2024-03-16T07:54:16.466 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T07:54:16.466 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274 --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/orig.config.yaml 2024-03-16T07:54:16.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T07:54:16.633 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274 branch: wip-batrick-testing-20240314.183243 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7602274' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnormpttfCKGWCKUOjckB27FFkBp2gjD+7haduh3HIhRdkrSE1tbj2Ur2LS2lD3F3ZYUDgAtPUf8+Z+v8+0RRY= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGub6YpicLRQ9cvdCDyu3L7E6X5nFxm4jOB7H8/H+kGiOMncWimwLT0XUrHYjFP0z6PiWpuaeuSWDoDR1tlZu00= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAZwaH2Q3LVkDBssky8FHxrHr/lqrLej7urkItChXsSmRfNhRY1nF/YDmTkXs+VAISpeWJg0ngFWVebfTTKx7uo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T07:54:16.634 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T07:54:16.634 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T07:54:16.634 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T07:54:16.636 INFO:teuthology.task.internal:Checking packages... 2024-03-16T07:54:16.660 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T07:54:16.660 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T07:54:16.660 INFO:teuthology.packaging:ref: None 2024-03-16T07:54:16.660 INFO:teuthology.packaging:tag: None 2024-03-16T07:54:16.660 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T07:54:16.660 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T07:54:16.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T07:54:16.882 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-16T07:54:16.883 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T07:54:16.893 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T07:54:16.894 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T07:54:16.984 INFO:teuthology.task.internal:Saving configuration 2024-03-16T07:54:17.006 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T07:54:17.013 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T07:54:17.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.351409', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnormpttfCKGWCKUOjckB27FFkBp2gjD+7haduh3HIhRdkrSE1tbj2Ur2LS2lD3F3ZYUDgAtPUf8+Z+v8+0RRY='} 2024-03-16T07:54:17.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.352525', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGub6YpicLRQ9cvdCDyu3L7E6X5nFxm4jOB7H8/H+kGiOMncWimwLT0XUrHYjFP0z6PiWpuaeuSWDoDR1tlZu00='} 2024-03-16T07:54:17.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.350200', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAZwaH2Q3LVkDBssky8FHxrHr/lqrLej7urkItChXsSmRfNhRY1nF/YDmTkXs+VAISpeWJg0ngFWVebfTTKx7uo='} 2024-03-16T07:54:17.085 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T07:54:17.092 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-16T07:54:17.092 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-16T07:54:17.092 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-16T07:54:17.092 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T07:54:17.191 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c7e0da160>, signals=[15]) 2024-03-16T07:54:17.192 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T07:54:17.198 INFO:teuthology.task.internal:Opening connections... 2024-03-16T07:54:17.198 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T07:54:17.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:54:17.310 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-03-16T07:54:17.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:54:17.408 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-03-16T07:54:17.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:54:17.497 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T07:54:17.504 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-03-16T07:54:17.522 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-03-16T07:54:17.522 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:54:17.580 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:54:17.581 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:54:17.581 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-03-16T07:54:17.601 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-03-16T07:54:17.619 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-03-16T07:54:17.619 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-03-16T07:54:17.676 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:54:17.677 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:54:17.677 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:54:17.677 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:54:17.677 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:54:17.677 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:54:17.677 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-03-16T07:54:17.701 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-03-16T07:54:17.719 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-03-16T07:54:17.719 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-03-16T07:54:17.776 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-03-16T07:54:17.776 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-03-16T07:54:17.776 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-03-16T07:54:17.776 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:54:17.776 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:54:17.777 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:54:17.777 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-03-16T07:54:17.801 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T07:54:17.811 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T07:54:17.827 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T07:54:17.828 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-03-16T07:54:17.830 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-03-16T07:54:17.832 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-03-16T07:54:17.849 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T07:54:17.854 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T07:54:17.854 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-03-16T07:54:17.887 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-03-16T07:54:17.890 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-03-16T07:54:17.922 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T07:54:18.002 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T07:54:18.019 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-16T07:54:18.019 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-16T07:54:18.019 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-16T07:54:18.019 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T07:54:18.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:54:18.037 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-16T07:54:18.095 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-361.el9.x86_64 2024-03-16T07:54:18.095 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-361.el9.x86_64 2024-03-16T07:54:18.095 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-03-16T07:54:20.299 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-16T07:54:21.849 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-16T07:54:24.298 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-03-16T07:54:26.090 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-03-16T07:54:31.233 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-16T07:54:32.725 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2024-03-16T07:54:33.576 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-03-16T07:54:38.701 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-16T07:54:40.983 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T07:54:40.983 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T07:54:41.070 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T07:54:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 100 M 2024-03-16T07:54:41.081 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 123 M 2024-03-16T07:54:41.082 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-16T07:54:43.037 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 3.2 MB/s | 6.0 MB 00:01 2024-03-16T07:54:47.327 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 3.4 MB/s | 21 MB 00:06 2024-03-16T07:55:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 1.4 MB/s | 39 MB 00:28 2024-03-16T07:55:12.283 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 1.1 MB/s | 33 MB 00:29 2024-03-16T07:55:12.283 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-16T07:55:12.283 INFO:teuthology.orchestra.run.smithi049.stdout:Total 3.2 MB/s | 100 MB 00:31 2024-03-16T07:55:13.208 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-16T07:55:13.647 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-16T07:55:13.647 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-16T07:55:15.057 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-16T07:55:15.058 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-16T07:55:17.922 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-16T07:55:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T07:55:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T07:55:21.677 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T07:55:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T07:55:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T07:55:27.074 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:55:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:56:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:56:02.755 INFO:teuthology.orchestra.run.smithi049.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T07:56:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:56:02.842 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:56:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:56:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T07:56:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T07:56:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T07:56:03.716 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T07:56:03.717 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T07:56:03.717 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:56:03.717 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-16T07:56:04.288 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T07:56:04.430 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T07:56:04.433 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T07:56:04.433 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T07:56:04.433 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T07:56:04.434 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T07:56:04.434 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-16T07:56:04.434 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T07:56:04.434 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T07:56:04.561 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T07:56:04.561 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T07:56:04.561 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-03-16T07:56:04.598 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-03-16T07:56:04.598 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-03-16T07:56:04.598 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-03-16T07:56:04.599 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T07:56:04.600 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T07:56:04.601 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T07:56:04.601 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-427.el9.x86_64 2024-03-16T07:56:04.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T07:56:05.196 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-03-16T07:56:06.681 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T07:56:06.703 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-03-16T07:56:06.705 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-03-16T07:56:06.720 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.4XR8yoaUE2 2024-03-16T07:56:06.720 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4XR8yoaUE2 2024-03-16T07:56:06.791 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.4XR8yoaUE2 2024-03-16T07:56:06.931 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.4XR8yoaUE2 is 6KB 2024-03-16T07:56:06.951 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.4XR8yoaUE2 2024-03-16T07:56:06.965 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T07:56:07.041 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T07:56:07.041 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T07:56:07.041 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T07:56:07.041 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T07:56:07.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T07:56:07.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-03-16T07:56:37.196 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-16T07:56:37.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:56:37.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:55.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-16T07:57:04.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:57:04.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:07.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-16T07:57:19.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:57:19.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:22.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-16T07:57:37.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:57:37.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:40.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-16T07:57:58.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:57:58.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:17.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-16T07:58:38.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:58:38.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:38.958 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-16T07:58:39.370 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T07:58:39.370 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-16T07:58:39.370 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-16T07:58:39.385 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-427.el9.x86_64 2024-03-16T07:58:39.385 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T07:58:39.385 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T07:58:39.385 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T07:58:40.386 DEBUG:teuthology.parallel:result is None 2024-03-16T07:58:40.387 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T07:58:40.395 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T07:58:40.395 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T07:58:40.397 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T07:58:40.400 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T07:58:40.417 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T07:58:40.425 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T07:58:40.442 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T07:58:40.443 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T07:58:40.456 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T07:58:40.458 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T07:58:40.566 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T07:58:40.575 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T07:58:40.575 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T07:58:40.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:58:40.592 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T07:58:40.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:58:40.609 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T07:58:40.612 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T07:58:40.634 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T07:58:40.675 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.683 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.691 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.695 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.703 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T07:58:40.746 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T07:58:40.754 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T07:58:40.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T07:58:40.757 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T07:58:40.759 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T07:58:40.790 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T07:58:40.799 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T07:58:40.799 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T07:58:40.832 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T07:58:40.835 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T07:58:40.850 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:40.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:40.990 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.080 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T07:58:41.080 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T07:58:41.152 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:41.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:41.248 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.332 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.377 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T07:58:41.377 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T07:58:41.446 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:41.491 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T07:58:41.549 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.631 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T07:58:41.678 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T07:58:41.678 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T07:58:41.747 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-03-16T07:58:41.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-03-16T07:58:41.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-03-16T07:58:41.791 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T07:58:41.808 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T07:58:41.828 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T07:58:42.252 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T07:58:42.260 INFO:teuthology.task.internal:Starting timer... 2024-03-16T07:58:42.260 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T07:58:42.313 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T07:58:42.328 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-16T07:58:42.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-03-16T07:58:42.391 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-03-16T07:58:42.394 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-03-16T07:58:42.469 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-03-16T07:58:42.471 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-03-16T07:58:42.535 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-03-16T07:58:42.537 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T07:58:42.537 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-03-16T07:58:42.557 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-03-16T07:58:42.558 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-03-16T07:58:42.598 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-03-16T07:58:42.598 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-03-16T07:58:42.619 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-03-16T07:58:42.619 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-03-16T07:58:42.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T07:58:42.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:58:42.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T07:58:42.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:58:42.681 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T07:58:42.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T07:58:42.713 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T07:58:42.713 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-03-16T07:58:42.753 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-03-16T07:58:42.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-03-16T07:58:42.829 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T07:58:42.839 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T07:58:42.860 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T07:58:42.862 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi129.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T08:03:51.212 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-03-16T08:03:51.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T08:03:51.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:03:51.321 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-16T08:03:51.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-16T08:03:51.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-03-16T08:03:51.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:03:51.505 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-03-16T08:03:51.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-03-16T08:03:51.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-16T08:03:51.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:03:51.663 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-03-16T08:03:51.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-16T08:03:51.729 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T08:03:51.742 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T08:03:51.742 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T08:03:51.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:03:51.745 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T08:03:51.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:03:51.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T08:03:51.748 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:03:51.778 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T08:03:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T08:03:51.795 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T08:03:51.798 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T08:03:51.799 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T08:03:51.815 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T08:03:51.824 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-03-16T08:03:51.836 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-03-16T08:03:51.838 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-03-16T08:03:51.843 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-03-16T08:03:51.853 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-03-16T08:03:51.854 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T08:03:51.857 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-03-16T08:03:51.870 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T08:03:51.872 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T08:03:51.873 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T08:03:51.889 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T08:03:51.892 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T08:03:51.914 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.917 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.933 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.937 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:03:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:03:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-16T08:03:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.961 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:03:51.961 INFO:teuthology.run_tasks:Running task install... 2024-03-16T08:03:51.970 DEBUG:teuthology.task.install:project ceph 2024-03-16T08:03:51.970 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T08:03:51.970 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T08:03:51.970 INFO:teuthology.task.install:Using flavor: default 2024-03-16T08:03:51.976 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T08:03:51.976 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T08:03:51.977 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T08:03:51.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:03:51.979 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T08:03:51.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:03:51.981 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T08:03:51.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:03:52.127 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T08:03:52.127 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T08:03:52.130 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T08:03:52.130 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T08:03:52.195 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T08:03:52.196 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T08:03:52.237 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T08:03:52.237 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:03:52.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T08:03:52.276 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T08:03:52.276 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T08:03:52.299 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T08:03:52.299 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:03:52.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T08:03:52.327 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T08:03:52.327 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T08:03:52.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T08:03:52.367 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T08:03:52.367 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:03:52.367 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T08:03:52.395 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T08:03:52.396 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T08:03:52.396 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T08:03:52.434 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T08:03:52.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T08:03:52.466 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-03-16T08:03:52.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-03-16T08:03:52.474 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T08:03:52.540 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-03-16T08:03:52.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-03-16T08:03:52.544 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T08:03:52.610 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-03-16T08:03:52.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-03-16T08:03:52.716 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-03-16T08:03:52.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T08:03:52.778 INFO:teuthology.orchestra.run.smithi154.stdout:74 files removed 2024-03-16T08:03:52.805 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T08:03:52.845 INFO:teuthology.orchestra.run.smithi129.stdout:74 files removed 2024-03-16T08:03:52.873 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T08:03:53.550 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-03-16T08:03:53.556 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 181 kB/s | 83 kB 00:00 2024-03-16T08:03:53.622 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 181 kB/s | 83 kB 00:00 2024-03-16T08:03:53.865 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 37 kB/s | 10 kB 00:00 2024-03-16T08:03:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-16T08:03:54.031 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-16T08:03:54.257 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-16T08:03:54.351 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-16T08:03:54.398 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-16T08:03:54.643 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-16T08:03:54.784 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-16T08:03:55.071 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.7 MB 00:00 2024-03-16T08:03:56.035 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-16T08:03:56.159 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-16T08:03:56.538 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-03-16T08:03:56.959 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-16T08:03:57.020 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-16T08:03:58.868 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-16T08:04:06.161 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 886 kB/s | 8.0 MB 00:09 2024-03-16T08:04:08.003 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-03-16T08:04:10.643 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 755 kB/s | 6.1 MB 00:08 2024-03-16T08:04:13.000 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-03-16T08:04:14.352 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 5.9 kB/s | 15 kB 00:02 2024-03-16T08:04:14.554 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-03-16T08:04:16.096 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-03-16T08:04:17.078 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-16T08:04:18.477 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2024-03-16T08:04:19.634 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 21 MB 00:03 2024-03-16T08:04:19.831 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-16T08:04:24.712 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-03-16T08:04:24.823 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-16T08:04:27.167 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.168 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.182 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T08:04:27.183 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.184 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T08:04:27.185 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.187 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.190 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.191 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T08:04:27.273 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-03-16T08:04:27.285 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-16T08:04:27.285 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T08:04:27.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T08:04:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T08:04:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T08:04:27.289 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T08:04:27.290 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T08:04:27.291 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T08:04:27.292 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T08:04:27.293 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T08:04:27.294 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T08:04:27.295 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T08:04:27.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T08:04:27.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T08:04:27.298 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:Install 150 Packages 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 24 Packages 2024-03-16T08:04:27.299 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:04:27.300 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 224 M 2024-03-16T08:04:27.300 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-03-16T08:04:27.438 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.439 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.454 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T08:04:27.455 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T08:04:27.457 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T08:04:27.458 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.460 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.463 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T08:04:27.464 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T08:04:27.552 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T08:04:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T08:04:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T08:04:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T08:04:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T08:04:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T08:04:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T08:04:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T08:04:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T08:04:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T08:04:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T08:04:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T08:04:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout:Install 150 Packages 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 24 Packages 2024-03-16T08:04:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:04:27.584 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 224 M 2024-03-16T08:04:27.584 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-16T08:04:28.230 INFO:teuthology.orchestra.run.smithi154.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-03-16T08:04:28.539 INFO:teuthology.orchestra.run.smithi154.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 2.8 MB/s | 890 kB 00:00 2024-03-16T08:04:28.609 INFO:teuthology.orchestra.run.smithi049.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-16T08:04:28.705 INFO:teuthology.orchestra.run.smithi154.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 870 kB/s | 145 kB 00:00 2024-03-16T08:04:29.072 INFO:teuthology.orchestra.run.smithi154.stdout:(4/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 5.1 MB/s | 5.4 MB 00:01 2024-03-16T08:04:29.281 INFO:teuthology.orchestra.run.smithi154.stdout:(5/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 4.1 MB/s | 2.4 MB 00:00 2024-03-16T08:04:29.356 INFO:teuthology.orchestra.run.smithi154.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-03-16T08:04:29.610 INFO:teuthology.orchestra.run.smithi049.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 890 kB/s | 890 kB 00:00 2024-03-16T08:04:29.681 INFO:teuthology.orchestra.run.smithi154.stdout:(7/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 12 MB/s | 21 MB 00:01 2024-03-16T08:04:29.760 INFO:teuthology.orchestra.run.smithi049.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 967 kB/s | 145 kB 00:00 2024-03-16T08:04:30.198 INFO:teuthology.orchestra.run.smithi154.stdout:(8/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-03-16T08:04:30.273 INFO:teuthology.orchestra.run.smithi154.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 337 kB/s | 25 kB 00:00 2024-03-16T08:04:30.940 INFO:teuthology.orchestra.run.smithi154.stdout:(10/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el 7.8 MB/s | 9.8 MB 00:01 2024-03-16T08:04:31.007 INFO:teuthology.orchestra.run.smithi154.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 3.3 MB/s | 221 kB 00:00 2024-03-16T08:04:31.049 INFO:teuthology.orchestra.run.smithi154.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 652 kB/s | 27 kB 00:00 2024-03-16T08:04:31.108 INFO:teuthology.orchestra.run.smithi154.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 543 kB/s | 32 kB 00:00 2024-03-16T08:04:31.176 INFO:teuthology.orchestra.run.smithi049.stdout:(4/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-03-16T08:04:31.225 INFO:teuthology.orchestra.run.smithi154.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 6.2 MB/s | 747 kB 00:00 2024-03-16T08:04:31.275 INFO:teuthology.orchestra.run.smithi154.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 3.2 MB/s | 163 kB 00:00 2024-03-16T08:04:31.326 INFO:teuthology.orchestra.run.smithi154.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 2.5 MB/s | 127 kB 00:00 2024-03-16T08:04:31.417 INFO:teuthology.orchestra.run.smithi154.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 5.4 MB/s | 501 kB 00:00 2024-03-16T08:04:31.727 INFO:teuthology.orchestra.run.smithi049.stdout:(5/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-03-16T08:04:31.893 INFO:teuthology.orchestra.run.smithi154.stdout:(18/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 6.6 MB/s | 17 MB 00:02 2024-03-16T08:04:31.961 INFO:teuthology.orchestra.run.smithi049.stdout:(6/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-03-16T08:04:31.968 INFO:teuthology.orchestra.run.smithi154.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 603 kB/s | 45 kB 00:00 2024-03-16T08:04:32.035 INFO:teuthology.orchestra.run.smithi154.stdout:(20/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 8.3 MB/s | 5.1 MB 00:00 2024-03-16T08:04:32.060 INFO:teuthology.orchestra.run.smithi154.stdout:(21/174): python3-ceph-common-19.0.0-2162.g3d78 1.4 MB/s | 131 kB 00:00 2024-03-16T08:04:32.085 INFO:teuthology.orchestra.run.smithi154.stdout:(22/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 3.2 MB/s | 162 kB 00:00 2024-03-16T08:04:32.136 INFO:teuthology.orchestra.run.smithi154.stdout:(23/174): python3-rados-19.0.0-2162.g3d78f5ba.e 4.2 MB/s | 320 kB 00:00 2024-03-16T08:04:32.161 INFO:teuthology.orchestra.run.smithi154.stdout:(24/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.9 MB/s | 298 kB 00:00 2024-03-16T08:04:32.194 INFO:teuthology.orchestra.run.smithi154.stdout:(25/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.7 MB/s | 99 kB 00:00 2024-03-16T08:04:32.211 INFO:teuthology.orchestra.run.smithi154.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-16T08:04:32.270 INFO:teuthology.orchestra.run.smithi154.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 2.8 MB/s | 170 kB 00:00 2024-03-16T08:04:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 681 kB/s | 34 kB 00:00 2024-03-16T08:04:32.379 INFO:teuthology.orchestra.run.smithi154.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 2.3 MB/s | 139 kB 00:00 2024-03-16T08:04:32.629 INFO:teuthology.orchestra.run.smithi154.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 7.2 MB/s | 1.8 MB 00:00 2024-03-16T08:04:32.680 INFO:teuthology.orchestra.run.smithi154.stdout:(31/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 6.4 MB/s | 3.1 MB 00:00 2024-03-16T08:04:32.772 INFO:teuthology.orchestra.run.smithi154.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.7 MB/s | 248 kB 00:00 2024-03-16T08:04:32.847 INFO:teuthology.orchestra.run.smithi154.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 658 kB/s | 49 kB 00:00 2024-03-16T08:04:32.861 INFO:teuthology.orchestra.run.smithi049.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-03-16T08:04:32.931 INFO:teuthology.orchestra.run.smithi154.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 197 kB/s | 16 kB 00:00 2024-03-16T08:04:33.014 INFO:teuthology.orchestra.run.smithi154.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.1 MB/s | 264 kB 00:00 2024-03-16T08:04:33.173 INFO:teuthology.orchestra.run.smithi154.stdout:(36/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-16T08:04:33.248 INFO:teuthology.orchestra.run.smithi154.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 243 kB/s | 18 kB 00:00 2024-03-16T08:04:33.424 INFO:teuthology.orchestra.run.smithi154.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 9.3 MB/s | 7.4 MB 00:00 2024-03-16T08:04:33.457 INFO:teuthology.orchestra.run.smithi154.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-16T08:04:33.499 INFO:teuthology.orchestra.run.smithi154.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-03-16T08:04:33.524 INFO:teuthology.orchestra.run.smithi154.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-16T08:04:33.550 INFO:teuthology.orchestra.run.smithi154.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-16T08:04:33.575 INFO:teuthology.orchestra.run.smithi154.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-16T08:04:33.650 INFO:teuthology.orchestra.run.smithi154.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-16T08:04:33.676 INFO:teuthology.orchestra.run.smithi154.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 175 kB/s | 17 kB 00:00 2024-03-16T08:04:33.701 INFO:teuthology.orchestra.run.smithi154.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-16T08:04:33.726 INFO:teuthology.orchestra.run.smithi154.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-16T08:04:33.751 INFO:teuthology.orchestra.run.smithi154.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-16T08:04:33.777 INFO:teuthology.orchestra.run.smithi154.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-16T08:04:33.810 INFO:teuthology.orchestra.run.smithi154.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-03-16T08:04:33.836 INFO:teuthology.orchestra.run.smithi154.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 804 kB/s | 46 kB 00:00 2024-03-16T08:04:33.861 INFO:teuthology.orchestra.run.smithi154.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-16T08:04:33.894 INFO:teuthology.orchestra.run.smithi154.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 187 kB 00:00 2024-03-16T08:04:33.920 INFO:teuthology.orchestra.run.smithi154.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-16T08:04:33.945 INFO:teuthology.orchestra.run.smithi154.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-16T08:04:34.037 INFO:teuthology.orchestra.run.smithi154.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-16T08:04:34.079 INFO:teuthology.orchestra.run.smithi154.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-16T08:04:34.105 INFO:teuthology.orchestra.run.smithi154.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-16T08:04:34.130 INFO:teuthology.orchestra.run.smithi154.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-16T08:04:34.255 INFO:teuthology.orchestra.run.smithi154.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-16T08:04:34.269 INFO:teuthology.orchestra.run.smithi049.stdout:(8/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 3.5 MB/s | 21 MB 00:05 2024-03-16T08:04:34.289 INFO:teuthology.orchestra.run.smithi154.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-03-16T08:04:34.306 INFO:teuthology.orchestra.run.smithi154.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-16T08:04:34.311 INFO:teuthology.orchestra.run.smithi049.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 608 kB/s | 25 kB 00:00 2024-03-16T08:04:34.431 INFO:teuthology.orchestra.run.smithi154.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-16T08:04:34.456 INFO:teuthology.orchestra.run.smithi154.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 164 kB/s | 24 kB 00:00 2024-03-16T08:04:34.482 INFO:teuthology.orchestra.run.smithi154.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-16T08:04:34.507 INFO:teuthology.orchestra.run.smithi154.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-16T08:04:34.526 INFO:teuthology.orchestra.run.smithi154.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 724 kB/s | 32 kB 00:00 2024-03-16T08:04:34.635 INFO:teuthology.orchestra.run.smithi154.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-16T08:04:34.645 INFO:teuthology.orchestra.run.smithi049.stdout:(10/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el 5.5 MB/s | 9.8 MB 00:01 2024-03-16T08:04:34.660 INFO:teuthology.orchestra.run.smithi154.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-16T08:04:34.685 INFO:teuthology.orchestra.run.smithi154.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-16T08:04:34.711 INFO:teuthology.orchestra.run.smithi154.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-16T08:04:34.711 INFO:teuthology.orchestra.run.smithi049.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 3.2 MB/s | 221 kB 00:00 2024-03-16T08:04:34.736 INFO:teuthology.orchestra.run.smithi154.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 814 kB/s | 41 kB 00:00 2024-03-16T08:04:34.762 INFO:teuthology.orchestra.run.smithi049.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 541 kB/s | 27 kB 00:00 2024-03-16T08:04:34.770 INFO:teuthology.orchestra.run.smithi154.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-16T08:04:34.795 INFO:teuthology.orchestra.run.smithi154.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-16T08:04:34.797 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-16T08:04:34.804 INFO:teuthology.orchestra.run.smithi049.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 766 kB/s | 32 kB 00:00 2024-03-16T08:04:34.821 INFO:teuthology.orchestra.run.smithi154.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-16T08:04:34.846 INFO:teuthology.orchestra.run.smithi154.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-03-16T08:04:34.871 INFO:teuthology.orchestra.run.smithi154.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-03-16T08:04:34.946 INFO:teuthology.orchestra.run.smithi049.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 5.2 MB/s | 747 kB 00:00 2024-03-16T08:04:34.996 INFO:teuthology.orchestra.run.smithi049.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 3.2 MB/s | 163 kB 00:00 2024-03-16T08:04:35.046 INFO:teuthology.orchestra.run.smithi049.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 2.5 MB/s | 127 kB 00:00 2024-03-16T08:04:35.147 INFO:teuthology.orchestra.run.smithi049.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 4.9 MB/s | 501 kB 00:00 2024-03-16T08:04:35.171 INFO:teuthology.orchestra.run.smithi154.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-16T08:04:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 116 kB/s | 39 kB 00:00 2024-03-16T08:04:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 701 kB/s | 41 kB 00:00 2024-03-16T08:04:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-03-16T08:04:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-16T08:04:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-16T08:04:35.331 INFO:teuthology.orchestra.run.smithi154.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-16T08:04:35.357 INFO:teuthology.orchestra.run.smithi154.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-16T08:04:35.382 INFO:teuthology.orchestra.run.smithi154.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T08:04:35.425 INFO:teuthology.orchestra.run.smithi154.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-16T08:04:35.449 INFO:teuthology.orchestra.run.smithi154.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-16T08:04:35.474 INFO:teuthology.orchestra.run.smithi154.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-16T08:04:35.500 INFO:teuthology.orchestra.run.smithi154.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-16T08:04:35.608 INFO:teuthology.orchestra.run.smithi154.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-03-16T08:04:35.650 INFO:teuthology.orchestra.run.smithi154.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-03-16T08:04:35.676 INFO:teuthology.orchestra.run.smithi154.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-03-16T08:04:35.701 INFO:teuthology.orchestra.run.smithi154.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-16T08:04:35.726 INFO:teuthology.orchestra.run.smithi154.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-03-16T08:04:35.751 INFO:teuthology.orchestra.run.smithi154.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-03-16T08:04:35.810 INFO:teuthology.orchestra.run.smithi154.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-03-16T08:04:35.830 INFO:teuthology.orchestra.run.smithi049.stdout:(18/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 7.5 MB/s | 5.1 MB 00:00 2024-03-16T08:04:35.852 INFO:teuthology.orchestra.run.smithi154.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-03-16T08:04:35.880 INFO:teuthology.orchestra.run.smithi049.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 906 kB/s | 45 kB 00:00 2024-03-16T08:04:35.903 INFO:teuthology.orchestra.run.smithi154.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-16T08:04:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 2.6 MB/s | 131 kB 00:00 2024-03-16T08:04:35.961 INFO:teuthology.orchestra.run.smithi154.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 18 MB/s | 4.1 MB 00:00 2024-03-16T08:04:35.978 INFO:teuthology.orchestra.run.smithi154.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 461 kB/s | 35 kB 00:00 2024-03-16T08:04:35.981 INFO:teuthology.orchestra.run.smithi049.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 3.2 MB/s | 162 kB 00:00 2024-03-16T08:04:36.004 INFO:teuthology.orchestra.run.smithi154.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-16T08:04:36.029 INFO:teuthology.orchestra.run.smithi154.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-16T08:04:36.064 INFO:teuthology.orchestra.run.smithi049.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.8 MB/s | 320 kB 00:00 2024-03-16T08:04:36.112 INFO:teuthology.orchestra.run.smithi154.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-16T08:04:36.148 INFO:teuthology.orchestra.run.smithi049.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.5 MB/s | 298 kB 00:00 2024-03-16T08:04:36.207 INFO:teuthology.orchestra.run.smithi049.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.7 MB/s | 99 kB 00:00 2024-03-16T08:04:36.221 INFO:teuthology.orchestra.run.smithi154.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-03-16T08:04:36.257 INFO:teuthology.orchestra.run.smithi049.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-16T08:04:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:(26/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 6.5 MB/s | 3.1 MB 00:00 2024-03-16T08:04:36.763 INFO:teuthology.orchestra.run.smithi154.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 843 kB/s | 548 kB 00:00 2024-03-16T08:04:36.774 INFO:teuthology.orchestra.run.smithi049.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 4.0 MB/s | 170 kB 00:00 2024-03-16T08:04:36.788 INFO:teuthology.orchestra.run.smithi154.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-03-16T08:04:36.816 INFO:teuthology.orchestra.run.smithi049.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 821 kB/s | 34 kB 00:00 2024-03-16T08:04:36.875 INFO:teuthology.orchestra.run.smithi049.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 2.3 MB/s | 139 kB 00:00 2024-03-16T08:04:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 7.2 MB/s | 1.8 MB 00:00 2024-03-16T08:04:37.389 INFO:teuthology.orchestra.run.smithi154.stdout:(108/174): ceph-test-19.0.0-2162.g3d78f5ba.el9. 6.7 MB/s | 48 MB 00:07 2024-03-16T08:04:37.414 INFO:teuthology.orchestra.run.smithi154.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 30 kB/s | 19 kB 00:00 2024-03-16T08:04:37.673 INFO:teuthology.orchestra.run.smithi154.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-03-16T08:04:37.781 INFO:teuthology.orchestra.run.smithi154.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 132 kB/s | 49 kB 00:00 2024-03-16T08:04:37.950 INFO:teuthology.orchestra.run.smithi049.stdout:(31/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 2.8 MB/s | 17 MB 00:05 2024-03-16T08:04:38.051 INFO:teuthology.orchestra.run.smithi049.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-03-16T08:04:38.073 INFO:teuthology.orchestra.run.smithi154.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 168 kB/s | 67 kB 00:00 2024-03-16T08:04:38.092 INFO:teuthology.orchestra.run.smithi049.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2162.g3d 1.7 MB/s | 248 kB 00:00 2024-03-16T08:04:38.118 INFO:teuthology.orchestra.run.smithi049.stdout:(34/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 739 kB/s | 49 kB 00:00 2024-03-16T08:04:38.151 INFO:teuthology.orchestra.run.smithi049.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2162.g3 281 kB/s | 16 kB 00:00 2024-03-16T08:04:38.185 INFO:teuthology.orchestra.run.smithi049.stdout:(36/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.9 MB/s | 264 kB 00:00 2024-03-16T08:04:38.227 INFO:teuthology.orchestra.run.smithi049.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 440 kB/s | 18 kB 00:00 2024-03-16T08:04:38.248 INFO:teuthology.orchestra.run.smithi154.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 324 kB/s | 151 kB 00:00 2024-03-16T08:04:38.324 INFO:teuthology.orchestra.run.smithi154.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-03-16T08:04:38.377 INFO:teuthology.orchestra.run.smithi049.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-03-16T08:04:38.461 INFO:teuthology.orchestra.run.smithi049.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-03-16T08:04:38.486 INFO:teuthology.orchestra.run.smithi049.stdout:(40/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 2.2 MB/s | 748 kB 00:00 2024-03-16T08:04:38.511 INFO:teuthology.orchestra.run.smithi049.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-16T08:04:38.537 INFO:teuthology.orchestra.run.smithi049.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-16T08:04:38.554 INFO:teuthology.orchestra.run.smithi049.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 768 kB/s | 32 kB 00:00 2024-03-16T08:04:38.629 INFO:teuthology.orchestra.run.smithi049.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-16T08:04:38.654 INFO:teuthology.orchestra.run.smithi049.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 174 kB/s | 17 kB 00:00 2024-03-16T08:04:38.679 INFO:teuthology.orchestra.run.smithi049.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-16T08:04:38.701 INFO:teuthology.orchestra.run.smithi154.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 116 kB/s | 43 kB 00:00 2024-03-16T08:04:38.705 INFO:teuthology.orchestra.run.smithi049.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-16T08:04:38.730 INFO:teuthology.orchestra.run.smithi049.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-16T08:04:38.732 INFO:teuthology.orchestra.run.smithi154.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-16T08:04:38.755 INFO:teuthology.orchestra.run.smithi049.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-16T08:04:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-03-16T08:04:38.814 INFO:teuthology.orchestra.run.smithi049.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 46 kB 00:00 2024-03-16T08:04:38.839 INFO:teuthology.orchestra.run.smithi049.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-16T08:04:38.865 INFO:teuthology.orchestra.run.smithi049.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-16T08:04:38.890 INFO:teuthology.orchestra.run.smithi049.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-16T08:04:38.891 INFO:teuthology.orchestra.run.smithi154.stdout:(117/174): python3-asyncssh-2.13.2-4.el9.noarch 853 kB/s | 548 kB 00:00 2024-03-16T08:04:38.915 INFO:teuthology.orchestra.run.smithi049.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-16T08:04:38.983 INFO:teuthology.orchestra.run.smithi154.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 113 kB/s | 32 kB 00:00 2024-03-16T08:04:39.007 INFO:teuthology.orchestra.run.smithi049.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-16T08:04:39.020 INFO:teuthology.orchestra.run.smithi154.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 52 kB/s | 15 kB 00:00 2024-03-16T08:04:39.049 INFO:teuthology.orchestra.run.smithi049.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-16T08:04:39.100 INFO:teuthology.orchestra.run.smithi049.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.9 MB/s | 811 kB 00:00 2024-03-16T08:04:39.125 INFO:teuthology.orchestra.run.smithi049.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.8 MB/s | 134 kB 00:00 2024-03-16T08:04:39.234 INFO:teuthology.orchestra.run.smithi049.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-16T08:04:39.267 INFO:teuthology.orchestra.run.smithi049.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.3 MB/s | 1.0 MB 00:00 2024-03-16T08:04:39.292 INFO:teuthology.orchestra.run.smithi049.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-16T08:04:39.362 INFO:teuthology.orchestra.run.smithi154.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 365 kB/s | 172 kB 00:00 2024-03-16T08:04:39.410 INFO:teuthology.orchestra.run.smithi049.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-16T08:04:39.434 INFO:teuthology.orchestra.run.smithi049.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-16T08:04:39.460 INFO:teuthology.orchestra.run.smithi049.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-16T08:04:39.485 INFO:teuthology.orchestra.run.smithi049.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-16T08:04:39.510 INFO:teuthology.orchestra.run.smithi049.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-03-16T08:04:39.529 INFO:teuthology.orchestra.run.smithi154.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 657 kB/s | 358 kB 00:00 2024-03-16T08:04:39.563 INFO:teuthology.orchestra.run.smithi154.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 559 kB/s | 303 kB 00:00 2024-03-16T08:04:39.663 INFO:teuthology.orchestra.run.smithi154.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 117 kB/s | 35 kB 00:00 2024-03-16T08:04:39.718 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-03-16T08:04:39.952 INFO:teuthology.orchestra.run.smithi049.stdout:(68/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 8.4 MB/s | 48 MB 00:05 2024-03-16T08:04:39.988 INFO:teuthology.orchestra.run.smithi154.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 441 kB/s | 203 kB 00:00 2024-03-16T08:04:40.019 INFO:teuthology.orchestra.run.smithi049.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.0 MB/s | 4.8 MB 00:00 2024-03-16T08:04:40.045 INFO:teuthology.orchestra.run.smithi049.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 409 kB/s | 217 kB 00:00 2024-03-16T08:04:40.070 INFO:teuthology.orchestra.run.smithi049.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2024-03-16T08:04:40.095 INFO:teuthology.orchestra.run.smithi049.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 835 kB/s | 41 kB 00:00 2024-03-16T08:04:40.120 INFO:teuthology.orchestra.run.smithi049.stdout:(73/174): python3-packaging-20.9-5.el9.noarch.r 919 kB/s | 69 kB 00:00 2024-03-16T08:04:40.121 INFO:teuthology.orchestra.run.smithi154.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 315 kB/s | 144 kB 00:00 2024-03-16T08:04:40.145 INFO:teuthology.orchestra.run.smithi049.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-16T08:04:40.171 INFO:teuthology.orchestra.run.smithi049.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-16T08:04:40.189 INFO:teuthology.orchestra.run.smithi154.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 53 kB/s | 11 kB 00:00 2024-03-16T08:04:40.196 INFO:teuthology.orchestra.run.smithi049.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-03-16T08:04:40.221 INFO:teuthology.orchestra.run.smithi049.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-16T08:04:40.238 INFO:teuthology.orchestra.run.smithi049.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 43 kB 00:00 2024-03-16T08:04:40.380 INFO:teuthology.orchestra.run.smithi049.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 279 kB/s | 39 kB 00:00 2024-03-16T08:04:40.405 INFO:teuthology.orchestra.run.smithi049.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 251 kB/s | 41 kB 00:00 2024-03-16T08:04:40.406 INFO:teuthology.orchestra.run.smithi154.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-16T08:04:40.431 INFO:teuthology.orchestra.run.smithi154.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 57 kB/s | 18 kB 00:00 2024-03-16T08:04:40.473 INFO:teuthology.orchestra.run.smithi154.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-03-16T08:04:40.581 INFO:teuthology.orchestra.run.smithi049.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-16T08:04:40.606 INFO:teuthology.orchestra.run.smithi049.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-03-16T08:04:40.623 INFO:teuthology.orchestra.run.smithi049.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 261 kB/s | 56 kB 00:00 2024-03-16T08:04:40.639 INFO:teuthology.orchestra.run.smithi049.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-16T08:04:40.656 INFO:teuthology.orchestra.run.smithi049.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-16T08:04:40.681 INFO:teuthology.orchestra.run.smithi049.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-16T08:04:40.690 INFO:teuthology.orchestra.run.smithi154.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 68 kB/s | 19 kB 00:00 2024-03-16T08:04:40.707 INFO:teuthology.orchestra.run.smithi049.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T08:04:40.731 INFO:teuthology.orchestra.run.smithi154.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 65 kB/s | 20 kB 00:00 2024-03-16T08:04:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-16T08:04:40.765 INFO:teuthology.orchestra.run.smithi049.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-16T08:04:40.782 INFO:teuthology.orchestra.run.smithi049.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-16T08:04:40.808 INFO:teuthology.orchestra.run.smithi049.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-16T08:04:41.050 INFO:teuthology.orchestra.run.smithi049.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 809 kB/s | 195 kB 00:00 2024-03-16T08:04:41.057 INFO:teuthology.orchestra.run.smithi154.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 126 kB/s | 46 kB 00:00 2024-03-16T08:04:41.075 INFO:teuthology.orchestra.run.smithi049.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 607 kB/s | 187 kB 00:00 2024-03-16T08:04:41.107 INFO:teuthology.orchestra.run.smithi154.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 210 kB/s | 79 kB 00:00 2024-03-16T08:04:41.117 INFO:teuthology.orchestra.run.smithi049.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-16T08:04:41.142 INFO:teuthology.orchestra.run.smithi049.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-16T08:04:41.176 INFO:teuthology.orchestra.run.smithi049.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-03-16T08:04:41.191 INFO:teuthology.orchestra.run.smithi154.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-03-16T08:04:41.201 INFO:teuthology.orchestra.run.smithi049.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 303 kB/s | 25 kB 00:00 2024-03-16T08:04:41.268 INFO:teuthology.orchestra.run.smithi049.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-16T08:04:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-03-16T08:04:41.318 INFO:teuthology.orchestra.run.smithi049.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-16T08:04:41.344 INFO:teuthology.orchestra.run.smithi049.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 694 kB/s | 35 kB 00:00 2024-03-16T08:04:41.369 INFO:teuthology.orchestra.run.smithi049.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-16T08:04:41.394 INFO:teuthology.orchestra.run.smithi049.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-16T08:04:41.468 INFO:teuthology.orchestra.run.smithi049.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-03-16T08:04:41.468 INFO:teuthology.orchestra.run.smithi154.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 157 kB/s | 58 kB 00:00 2024-03-16T08:04:41.468 INFO:teuthology.orchestra.run.smithi154.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 60 kB/s | 16 kB 00:00 2024-03-16T08:04:41.561 INFO:teuthology.orchestra.run.smithi049.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.9 MB/s | 4.1 MB 00:00 2024-03-16T08:04:41.637 INFO:teuthology.orchestra.run.smithi049.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-03-16T08:04:41.650 INFO:teuthology.orchestra.run.smithi154.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 502 kB/s | 272 kB 00:00 2024-03-16T08:04:41.767 INFO:teuthology.orchestra.run.smithi154.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 94 kB/s | 28 kB 00:00 2024-03-16T08:04:41.778 INFO:teuthology.orchestra.run.smithi049.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 138 kB/s | 19 kB 00:00 2024-03-16T08:04:41.800 INFO:teuthology.orchestra.run.smithi154.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 239 kB/s | 90 kB 00:00 2024-03-16T08:04:41.804 INFO:teuthology.orchestra.run.smithi049.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-16T08:04:41.829 INFO:teuthology.orchestra.run.smithi049.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-16T08:04:41.854 INFO:teuthology.orchestra.run.smithi049.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-03-16T08:04:41.880 INFO:teuthology.orchestra.run.smithi049.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-03-16T08:04:41.913 INFO:teuthology.orchestra.run.smithi049.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-16T08:04:41.934 INFO:teuthology.orchestra.run.smithi154.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 108 kB/s | 31 kB 00:00 2024-03-16T08:04:41.955 INFO:teuthology.orchestra.run.smithi049.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-16T08:04:42.130 INFO:teuthology.orchestra.run.smithi049.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-03-16T08:04:42.156 INFO:teuthology.orchestra.run.smithi049.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-03-16T08:04:42.176 INFO:teuthology.orchestra.run.smithi154.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 158 kB/s | 59 kB 00:00 2024-03-16T08:04:42.181 INFO:teuthology.orchestra.run.smithi049.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-03-16T08:04:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-03-16T08:04:42.207 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:42.208 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T08:04:42.223 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T08:04:42.224 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T08:04:42.225 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T08:04:42.226 INFO:teuthology.orchestra.run.smithi154.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 411 kB/s | 188 kB 00:00 2024-03-16T08:04:42.226 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:42.228 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T08:04:42.231 INFO:teuthology.orchestra.run.smithi049.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-16T08:04:42.232 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T08:04:42.233 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T08:04:42.243 INFO:teuthology.orchestra.run.smithi154.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 116 kB/s | 36 kB 00:00 2024-03-16T08:04:42.290 INFO:teuthology.orchestra.run.smithi049.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-16T08:04:42.313 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-03-16T08:04:42.333 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================================ 2024-03-16T08:04:42.333 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================================ 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T08:04:42.334 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T08:04:42.335 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T08:04:42.336 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T08:04:42.337 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T08:04:42.338 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T08:04:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T08:04:42.340 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T08:04:42.341 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T08:04:42.342 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T08:04:42.343 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T08:04:42.344 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T08:04:42.345 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================================ 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout:Install 150 Packages 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 24 Packages 2024-03-16T08:04:42.346 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:04:42.347 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 224 M 2024-03-16T08:04:42.347 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-03-16T08:04:42.382 INFO:teuthology.orchestra.run.smithi049.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-16T08:04:42.424 INFO:teuthology.orchestra.run.smithi049.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-03-16T08:04:42.450 INFO:teuthology.orchestra.run.smithi049.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 524 kB/s | 35 kB 00:00 2024-03-16T08:04:42.508 INFO:teuthology.orchestra.run.smithi049.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-16T08:04:42.543 INFO:teuthology.orchestra.run.smithi154.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 139 kB/s | 51 kB 00:00 2024-03-16T08:04:42.567 INFO:teuthology.orchestra.run.smithi049.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-16T08:04:42.609 INFO:teuthology.orchestra.run.smithi049.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-03-16T08:04:42.618 INFO:teuthology.orchestra.run.smithi154.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 239 kB/s | 90 kB 00:00 2024-03-16T08:04:42.651 INFO:teuthology.orchestra.run.smithi049.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-03-16T08:04:42.684 INFO:teuthology.orchestra.run.smithi049.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 701 kB/s | 23 kB 00:00 2024-03-16T08:04:42.718 INFO:teuthology.orchestra.run.smithi049.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 584 kB/s | 19 kB 00:00 2024-03-16T08:04:42.751 INFO:teuthology.orchestra.run.smithi049.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-03-16T08:04:42.777 INFO:teuthology.orchestra.run.smithi154.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 417 kB/s | 230 kB 00:00 2024-03-16T08:04:42.902 INFO:teuthology.orchestra.run.smithi154.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 70 kB/s | 20 kB 00:00 2024-03-16T08:04:43.019 INFO:teuthology.orchestra.run.smithi129.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-03-16T08:04:43.044 INFO:teuthology.orchestra.run.smithi049.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-03-16T08:04:43.077 INFO:teuthology.orchestra.run.smithi049.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-03-16T08:04:43.136 INFO:teuthology.orchestra.run.smithi049.stdout:(132/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-03-16T08:04:43.161 INFO:teuthology.orchestra.run.smithi049.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-03-16T08:04:43.178 INFO:teuthology.orchestra.run.smithi049.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-03-16T08:04:43.194 INFO:teuthology.orchestra.run.smithi154.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 657 kB/s | 427 kB 00:00 2024-03-16T08:04:43.212 INFO:teuthology.orchestra.run.smithi049.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 491 kB/s | 16 kB 00:00 2024-03-16T08:04:43.236 INFO:teuthology.orchestra.run.smithi154.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 418 kB/s | 192 kB 00:00 2024-03-16T08:04:43.270 INFO:teuthology.orchestra.run.smithi049.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-16T08:04:43.311 INFO:teuthology.orchestra.run.smithi129.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 3.0 MB/s | 890 kB 00:00 2024-03-16T08:04:43.312 INFO:teuthology.orchestra.run.smithi049.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 680 kB/s | 28 kB 00:00 2024-03-16T08:04:43.337 INFO:teuthology.orchestra.run.smithi049.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-03-16T08:04:43.363 INFO:teuthology.orchestra.run.smithi049.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-16T08:04:43.405 INFO:teuthology.orchestra.run.smithi049.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-16T08:04:43.429 INFO:teuthology.orchestra.run.smithi129.stdout:(3/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 8.4 MB/s | 5.4 MB 00:00 2024-03-16T08:04:43.447 INFO:teuthology.orchestra.run.smithi049.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 859 kB/s | 36 kB 00:00 2024-03-16T08:04:43.454 INFO:teuthology.orchestra.run.smithi129.stdout:(4/174): ceph-immutable-object-cache-19.0.0-216 1.0 MB/s | 145 kB 00:00 2024-03-16T08:04:43.488 INFO:teuthology.orchestra.run.smithi049.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-16T08:04:43.514 INFO:teuthology.orchestra.run.smithi049.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-03-16T08:04:43.673 INFO:teuthology.orchestra.run.smithi049.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 566 kB/s | 90 kB 00:00 2024-03-16T08:04:43.720 INFO:teuthology.orchestra.run.smithi154.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-03-16T08:04:43.720 INFO:teuthology.orchestra.run.smithi129.stdout:(5/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-03-16T08:04:43.723 INFO:teuthology.orchestra.run.smithi049.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 982 kB/s | 230 kB 00:00 2024-03-16T08:04:43.762 INFO:teuthology.orchestra.run.smithi154.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-16T08:04:43.765 INFO:teuthology.orchestra.run.smithi049.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-03-16T08:04:43.882 INFO:teuthology.orchestra.run.smithi049.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-03-16T08:04:43.889 INFO:teuthology.orchestra.run.smithi154.stdout:(152/174): librados2-19.0.0-2162.g3d78f5ba.el9. 4.9 MB/s | 3.4 MB 00:00 2024-03-16T08:04:43.907 INFO:teuthology.orchestra.run.smithi049.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-16T08:04:43.921 INFO:teuthology.orchestra.run.smithi129.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-03-16T08:04:44.023 INFO:teuthology.orchestra.run.smithi154.stdout:(153/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-03-16T08:04:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.2 MB/s | 2.3 MB 00:00 2024-03-16T08:04:44.090 INFO:teuthology.orchestra.run.smithi049.stdout:(149/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.2 MB/s | 2.0 MB 00:01 2024-03-16T08:04:44.107 INFO:teuthology.orchestra.run.smithi154.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 508 kB/s | 42 kB 00:00 2024-03-16T08:04:44.132 INFO:teuthology.orchestra.run.smithi154.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-16T08:04:44.163 INFO:teuthology.orchestra.run.smithi129.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-16T08:04:44.182 INFO:teuthology.orchestra.run.smithi154.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 6.2 MB/s | 1.8 MB 00:00 2024-03-16T08:04:44.199 INFO:teuthology.orchestra.run.smithi154.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-16T08:04:44.224 INFO:teuthology.orchestra.run.smithi154.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 609 kB/s | 71 kB 00:00 2024-03-16T08:04:44.250 INFO:teuthology.orchestra.run.smithi154.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-16T08:04:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 548 kB/s | 23 kB 00:00 2024-03-16T08:04:44.292 INFO:teuthology.orchestra.run.smithi154.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-16T08:04:44.317 INFO:teuthology.orchestra.run.smithi154.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-16T08:04:44.342 INFO:teuthology.orchestra.run.smithi154.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-16T08:04:44.384 INFO:teuthology.orchestra.run.smithi154.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-03-16T08:04:44.401 INFO:teuthology.orchestra.run.smithi154.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 764 kB/s | 64 kB 00:00 2024-03-16T08:04:44.418 INFO:teuthology.orchestra.run.smithi154.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-16T08:04:44.460 INFO:teuthology.orchestra.run.smithi154.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-03-16T08:04:44.502 INFO:teuthology.orchestra.run.smithi154.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 21 MB/s | 2.2 MB 00:00 2024-03-16T08:04:44.527 INFO:teuthology.orchestra.run.smithi154.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-16T08:04:44.544 INFO:teuthology.orchestra.run.smithi154.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 745 kB/s | 31 kB 00:00 2024-03-16T08:04:44.594 INFO:teuthology.orchestra.run.smithi154.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-16T08:04:44.620 INFO:teuthology.orchestra.run.smithi154.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 990 kB/s | 74 kB 00:00 2024-03-16T08:04:44.653 INFO:teuthology.orchestra.run.smithi154.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-03-16T08:04:44.657 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:04:44.657 INFO:teuthology.orchestra.run.smithi154.stdout:Total 13 MB/s | 224 MB 00:17 2024-03-16T08:04:44.666 INFO:teuthology.orchestra.run.smithi049.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-03-16T08:04:44.800 INFO:teuthology.orchestra.run.smithi049.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.3 MB/s | 321 kB 00:00 2024-03-16T08:04:45.080 INFO:teuthology.orchestra.run.smithi129.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 11 MB/s | 9.8 MB 00:00 2024-03-16T08:04:45.108 INFO:teuthology.orchestra.run.smithi049.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.4 MB/s | 2.3 MB 00:00 2024-03-16T08:04:45.115 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-03-16T08:04:45.138 INFO:teuthology.orchestra.run.smithi129.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 433 kB/s | 25 kB 00:00 2024-03-16T08:04:45.200 INFO:teuthology.orchestra.run.smithi049.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-16T08:04:45.259 INFO:teuthology.orchestra.run.smithi049.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 727 kB/s | 42 kB 00:00 2024-03-16T08:04:45.317 INFO:teuthology.orchestra.run.smithi049.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-16T08:04:45.368 INFO:teuthology.orchestra.run.smithi049.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-16T08:04:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-03-16T08:04:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-03-16T08:04:45.426 INFO:teuthology.orchestra.run.smithi049.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 625 kB/s | 36 kB 00:00 2024-03-16T08:04:45.485 INFO:teuthology.orchestra.run.smithi049.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-16T08:04:45.543 INFO:teuthology.orchestra.run.smithi049.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-16T08:04:45.610 INFO:teuthology.orchestra.run.smithi049.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-03-16T08:04:45.669 INFO:teuthology.orchestra.run.smithi049.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-16T08:04:45.727 INFO:teuthology.orchestra.run.smithi049.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-16T08:04:45.786 INFO:teuthology.orchestra.run.smithi049.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-16T08:04:45.845 INFO:teuthology.orchestra.run.smithi049.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-16T08:04:45.903 INFO:teuthology.orchestra.run.smithi049.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 493 kB/s | 29 kB 00:00 2024-03-16T08:04:45.978 INFO:teuthology.orchestra.run.smithi049.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-03-16T08:04:46.070 INFO:teuthology.orchestra.run.smithi049.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-03-16T08:04:46.180 INFO:teuthology.orchestra.run.smithi129.stdout:(10/174): ceph-common-19.0.0-2162.g3d78f5ba.el9 6.1 MB/s | 21 MB 00:03 2024-03-16T08:04:46.196 INFO:teuthology.orchestra.run.smithi049.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.3 MB 00:00 2024-03-16T08:04:46.263 INFO:teuthology.orchestra.run.smithi049.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-16T08:04:46.297 INFO:teuthology.orchestra.run.smithi129.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 1.9 MB/s | 221 kB 00:00 2024-03-16T08:04:46.313 INFO:teuthology.orchestra.run.smithi049.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 625 kB/s | 31 kB 00:00 2024-03-16T08:04:46.396 INFO:teuthology.orchestra.run.smithi049.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-16T08:04:46.505 INFO:teuthology.orchestra.run.smithi049.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 688 kB/s | 74 kB 00:00 2024-03-16T08:04:46.531 INFO:teuthology.orchestra.run.smithi129.stdout:(12/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 6.5 MB/s | 17 MB 00:02 2024-03-16T08:04:46.556 INFO:teuthology.orchestra.run.smithi129.stdout:(13/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 105 kB/s | 27 kB 00:00 2024-03-16T08:04:46.606 INFO:teuthology.orchestra.run.smithi129.stdout:(14/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 423 kB/s | 32 kB 00:00 2024-03-16T08:04:46.715 INFO:teuthology.orchestra.run.smithi129.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.5 MB/s | 163 kB 00:00 2024-03-16T08:04:46.748 INFO:teuthology.orchestra.run.smithi129.stdout:(16/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 3.8 MB/s | 747 kB 00:00 2024-03-16T08:04:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:(173/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 1.2 MB/s | 3.1 MB 00:02 2024-03-16T08:04:46.782 INFO:teuthology.orchestra.run.smithi129.stdout:(17/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.8 MB/s | 127 kB 00:00 2024-03-16T08:04:46.857 INFO:teuthology.orchestra.run.smithi129.stdout:(18/174): libradosstriper1-19.0.0-2162.g3d78f5b 4.5 MB/s | 501 kB 00:00 2024-03-16T08:04:46.957 INFO:teuthology.orchestra.run.smithi129.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 452 kB/s | 45 kB 00:00 2024-03-16T08:04:47.066 INFO:teuthology.orchestra.run.smithi129.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.2 MB/s | 131 kB 00:00 2024-03-16T08:04:47.100 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-03-16T08:04:47.100 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-03-16T08:04:47.130 INFO:teuthology.orchestra.run.smithi049.stdout:(174/174): librados2-19.0.0-2162.g3d78f5ba.el9. 1.1 MB/s | 3.4 MB 00:03 2024-03-16T08:04:47.138 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:04:47.139 INFO:teuthology.orchestra.run.smithi049.stdout:Total 11 MB/s | 224 MB 00:19 2024-03-16T08:04:47.166 INFO:teuthology.orchestra.run.smithi129.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.6 MB/s | 162 kB 00:00 2024-03-16T08:04:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 2.7 MB/s | 320 kB 00:00 2024-03-16T08:04:47.400 INFO:teuthology.orchestra.run.smithi129.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 2.5 MB/s | 298 kB 00:00 2024-03-16T08:04:47.517 INFO:teuthology.orchestra.run.smithi129.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 854 kB/s | 99 kB 00:00 2024-03-16T08:04:47.634 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-16T08:04:47.642 INFO:teuthology.orchestra.run.smithi129.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 683 kB/s | 85 kB 00:00 2024-03-16T08:04:47.976 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-16T08:04:47.976 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-16T08:04:48.010 INFO:teuthology.orchestra.run.smithi129.stdout:(26/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 4.2 MB/s | 5.1 MB 00:01 2024-03-16T08:04:48.109 INFO:teuthology.orchestra.run.smithi129.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-16T08:04:48.202 INFO:teuthology.orchestra.run.smithi129.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 370 kB/s | 34 kB 00:00 2024-03-16T08:04:48.260 INFO:teuthology.orchestra.run.smithi129.stdout:(29/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 5.0 MB/s | 3.1 MB 00:00 2024-03-16T08:04:48.294 INFO:teuthology.orchestra.run.smithi129.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.5 MB/s | 139 kB 00:00 2024-03-16T08:04:48.678 INFO:teuthology.orchestra.run.smithi129.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 4.3 MB/s | 1.8 MB 00:00 2024-03-16T08:04:48.786 INFO:teuthology.orchestra.run.smithi129.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.2 MB/s | 248 kB 00:00 2024-03-16T08:04:48.887 INFO:teuthology.orchestra.run.smithi129.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 492 kB/s | 49 kB 00:00 2024-03-16T08:04:48.979 INFO:teuthology.orchestra.run.smithi129.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 179 kB/s | 16 kB 00:00 2024-03-16T08:04:49.087 INFO:teuthology.orchestra.run.smithi129.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 2.4 MB/s | 264 kB 00:00 2024-03-16T08:04:49.287 INFO:teuthology.orchestra.run.smithi129.stdout:(36/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 3.7 MB/s | 748 kB 00:00 2024-03-16T08:04:49.371 INFO:teuthology.orchestra.run.smithi129.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 219 kB/s | 18 kB 00:00 2024-03-16T08:04:49.513 INFO:teuthology.orchestra.run.smithi129.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-16T08:04:49.571 INFO:teuthology.orchestra.run.smithi129.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-16T08:04:49.605 INFO:teuthology.orchestra.run.smithi129.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-16T08:04:49.647 INFO:teuthology.orchestra.run.smithi129.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-16T08:04:49.672 INFO:teuthology.orchestra.run.smithi129.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-03-16T08:04:49.764 INFO:teuthology.orchestra.run.smithi129.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-16T08:04:49.790 INFO:teuthology.orchestra.run.smithi129.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 688 kB/s | 17 kB 00:00 2024-03-16T08:04:49.823 INFO:teuthology.orchestra.run.smithi129.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-16T08:04:49.865 INFO:teuthology.orchestra.run.smithi129.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-16T08:04:49.899 INFO:teuthology.orchestra.run.smithi129.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-16T08:04:49.941 INFO:teuthology.orchestra.run.smithi129.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-16T08:04:49.991 INFO:teuthology.orchestra.run.smithi129.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-16T08:04:50.025 INFO:teuthology.orchestra.run.smithi129.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-16T08:04:50.067 INFO:teuthology.orchestra.run.smithi129.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-16T08:04:50.090 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-16T08:04:50.091 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-16T08:04:50.108 INFO:teuthology.orchestra.run.smithi129.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-03-16T08:04:50.201 INFO:teuthology.orchestra.run.smithi129.stdout:(53/174): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-03-16T08:04:50.226 INFO:teuthology.orchestra.run.smithi129.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-03-16T08:04:50.251 INFO:teuthology.orchestra.run.smithi129.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 690 kB/s | 34 kB 00:00 2024-03-16T08:04:50.351 INFO:teuthology.orchestra.run.smithi129.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-16T08:04:50.385 INFO:teuthology.orchestra.run.smithi129.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-16T08:04:50.421 INFO:teuthology.orchestra.run.smithi129.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-16T08:04:50.446 INFO:teuthology.orchestra.run.smithi129.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-16T08:04:50.555 INFO:teuthology.orchestra.run.smithi129.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-16T08:04:50.589 INFO:teuthology.orchestra.run.smithi129.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-03-16T08:04:50.614 INFO:teuthology.orchestra.run.smithi129.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-16T08:04:50.748 INFO:teuthology.orchestra.run.smithi129.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-16T08:04:50.765 INFO:teuthology.orchestra.run.smithi129.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-16T08:04:50.782 INFO:teuthology.orchestra.run.smithi129.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 539 kB/s | 18 kB 00:00 2024-03-16T08:04:50.807 INFO:teuthology.orchestra.run.smithi129.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-16T08:04:50.833 INFO:teuthology.orchestra.run.smithi129.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-16T08:04:50.941 INFO:teuthology.orchestra.run.smithi129.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-16T08:04:50.967 INFO:teuthology.orchestra.run.smithi129.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-16T08:04:50.992 INFO:teuthology.orchestra.run.smithi129.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-16T08:04:51.017 INFO:teuthology.orchestra.run.smithi129.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-16T08:04:51.042 INFO:teuthology.orchestra.run.smithi129.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-16T08:04:51.067 INFO:teuthology.orchestra.run.smithi129.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-16T08:04:51.093 INFO:teuthology.orchestra.run.smithi129.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-16T08:04:51.118 INFO:teuthology.orchestra.run.smithi129.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-16T08:04:51.144 INFO:teuthology.orchestra.run.smithi129.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 941 kB/s | 47 kB 00:00 2024-03-16T08:04:51.169 INFO:teuthology.orchestra.run.smithi129.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 848 kB/s | 43 kB 00:00 2024-03-16T08:04:51.319 INFO:teuthology.orchestra.run.smithi129.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 261 kB/s | 39 kB 00:00 2024-03-16T08:04:51.478 INFO:teuthology.orchestra.run.smithi129.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-16T08:04:51.503 INFO:teuthology.orchestra.run.smithi129.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 226 kB/s | 41 kB 00:00 2024-03-16T08:04:51.528 INFO:teuthology.orchestra.run.smithi129.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-16T08:04:51.553 INFO:teuthology.orchestra.run.smithi129.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-16T08:04:51.579 INFO:teuthology.orchestra.run.smithi129.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-16T08:04:51.604 INFO:teuthology.orchestra.run.smithi129.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-16T08:04:51.621 INFO:teuthology.orchestra.run.smithi129.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-16T08:04:51.646 INFO:teuthology.orchestra.run.smithi129.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-16T08:04:51.680 INFO:teuthology.orchestra.run.smithi129.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-16T08:04:51.705 INFO:teuthology.orchestra.run.smithi129.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-16T08:04:51.730 INFO:teuthology.orchestra.run.smithi129.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-16T08:04:51.772 INFO:teuthology.orchestra.run.smithi129.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-03-16T08:04:51.881 INFO:teuthology.orchestra.run.smithi129.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-16T08:04:51.931 INFO:teuthology.orchestra.run.smithi129.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-03-16T08:04:51.965 INFO:teuthology.orchestra.run.smithi129.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-16T08:04:51.990 INFO:teuthology.orchestra.run.smithi129.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-16T08:04:52.082 INFO:teuthology.orchestra.run.smithi129.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-03-16T08:04:52.099 INFO:teuthology.orchestra.run.smithi129.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-16T08:04:52.258 INFO:teuthology.orchestra.run.smithi129.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-16T08:04:52.325 INFO:teuthology.orchestra.run.smithi129.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-03-16T08:04:52.392 INFO:teuthology.orchestra.run.smithi129.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-16T08:04:52.459 INFO:teuthology.orchestra.run.smithi129.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 523 kB/s | 35 kB 00:00 2024-03-16T08:04:52.526 INFO:teuthology.orchestra.run.smithi129.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.3 MB/s | 4.1 MB 00:00 2024-03-16T08:04:52.542 INFO:teuthology.orchestra.run.smithi129.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 534 kB/s | 44 kB 00:00 2024-03-16T08:04:52.584 INFO:teuthology.orchestra.run.smithi129.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-16T08:04:52.601 INFO:teuthology.orchestra.run.smithi129.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-03-16T08:04:52.693 INFO:teuthology.orchestra.run.smithi129.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-03-16T08:04:52.777 INFO:teuthology.orchestra.run.smithi129.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-03-16T08:04:52.819 INFO:teuthology.orchestra.run.smithi129.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-03-16T08:04:52.852 INFO:teuthology.orchestra.run.smithi129.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-03-16T08:04:52.895 INFO:teuthology.orchestra.run.smithi129.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 617 kB/s | 26 kB 00:00 2024-03-16T08:04:52.945 INFO:teuthology.orchestra.run.smithi129.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-03-16T08:04:53.004 INFO:teuthology.orchestra.run.smithi129.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-03-16T08:04:53.029 INFO:teuthology.orchestra.run.smithi129.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-03-16T08:04:53.063 INFO:teuthology.orchestra.run.smithi129.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-16T08:04:53.121 INFO:teuthology.orchestra.run.smithi129.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-16T08:04:53.155 INFO:teuthology.orchestra.run.smithi129.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-16T08:04:53.180 INFO:teuthology.orchestra.run.smithi129.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-03-16T08:04:53.205 INFO:teuthology.orchestra.run.smithi129.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-03-16T08:04:53.231 INFO:teuthology.orchestra.run.smithi129.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-16T08:04:53.256 INFO:teuthology.orchestra.run.smithi129.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-16T08:04:53.289 INFO:teuthology.orchestra.run.smithi129.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-03-16T08:04:53.315 INFO:teuthology.orchestra.run.smithi129.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-03-16T08:04:53.340 INFO:teuthology.orchestra.run.smithi129.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-03-16T08:04:53.365 INFO:teuthology.orchestra.run.smithi129.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-03-16T08:04:53.416 INFO:teuthology.orchestra.run.smithi129.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-03-16T08:04:53.458 INFO:teuthology.orchestra.run.smithi129.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-03-16T08:04:53.483 INFO:teuthology.orchestra.run.smithi129.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-03-16T08:04:53.508 INFO:teuthology.orchestra.run.smithi129.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-16T08:04:53.534 INFO:teuthology.orchestra.run.smithi129.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-03-16T08:04:53.559 INFO:teuthology.orchestra.run.smithi129.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-03-16T08:04:53.585 INFO:teuthology.orchestra.run.smithi129.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-03-16T08:04:53.626 INFO:teuthology.orchestra.run.smithi129.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-03-16T08:04:53.668 INFO:teuthology.orchestra.run.smithi129.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-03-16T08:04:53.693 INFO:teuthology.orchestra.run.smithi129.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-16T08:04:53.719 INFO:teuthology.orchestra.run.smithi129.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-16T08:04:53.744 INFO:teuthology.orchestra.run.smithi129.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-16T08:04:53.769 INFO:teuthology.orchestra.run.smithi129.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-16T08:04:53.795 INFO:teuthology.orchestra.run.smithi129.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-03-16T08:04:53.812 INFO:teuthology.orchestra.run.smithi129.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-03-16T08:04:53.837 INFO:teuthology.orchestra.run.smithi129.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 735 kB/s | 31 kB 00:00 2024-03-16T08:04:53.862 INFO:teuthology.orchestra.run.smithi129.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-16T08:04:53.879 INFO:teuthology.orchestra.run.smithi129.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-16T08:04:53.904 INFO:teuthology.orchestra.run.smithi129.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-03-16T08:04:53.921 INFO:teuthology.orchestra.run.smithi129.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-16T08:04:53.955 INFO:teuthology.orchestra.run.smithi129.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-16T08:04:53.980 INFO:teuthology.orchestra.run.smithi129.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-16T08:04:54.005 INFO:teuthology.orchestra.run.smithi129.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-03-16T08:04:54.031 INFO:teuthology.orchestra.run.smithi129.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-03-16T08:04:54.056 INFO:teuthology.orchestra.run.smithi129.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-16T08:04:54.070 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-03-16T08:04:54.098 INFO:teuthology.orchestra.run.smithi129.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-03-16T08:04:54.182 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T08:04:54.216 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T08:04:54.247 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:04:54.256 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:04:54.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T08:04:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T08:04:54.392 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T08:04:54.418 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T08:04:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T08:04:54.779 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T08:04:54.835 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:04:54.940 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:04:54.969 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:04:55.060 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:04:55.090 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:04:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:(150/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-03-16T08:04:55.137 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T08:04:55.173 INFO:teuthology.orchestra.run.smithi129.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-16T08:04:55.304 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T08:04:55.357 INFO:teuthology.orchestra.run.smithi129.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-16T08:04:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T08:04:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:04:55.424 INFO:teuthology.orchestra.run.smithi129.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-03-16T08:04:55.466 INFO:teuthology.orchestra.run.smithi129.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-16T08:04:55.494 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:04:55.509 INFO:teuthology.orchestra.run.smithi129.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-16T08:04:55.517 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:04:55.550 INFO:teuthology.orchestra.run.smithi129.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-16T08:04:55.564 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:04:55.592 INFO:teuthology.orchestra.run.smithi129.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 879 kB/s | 36 kB 00:00 2024-03-16T08:04:55.604 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T08:04:55.627 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:04:55.634 INFO:teuthology.orchestra.run.smithi129.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-16T08:04:55.642 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:04:55.676 INFO:teuthology.orchestra.run.smithi129.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-16T08:04:55.718 INFO:teuthology.orchestra.run.smithi129.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 553 kB/s | 23 kB 00:00 2024-03-16T08:04:55.760 INFO:teuthology.orchestra.run.smithi129.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-16T08:04:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:04:55.801 INFO:teuthology.orchestra.run.smithi129.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-03-16T08:04:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:04:55.844 INFO:teuthology.orchestra.run.smithi129.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-16T08:04:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T08:04:55.885 INFO:teuthology.orchestra.run.smithi129.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-16T08:04:55.927 INFO:teuthology.orchestra.run.smithi129.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 687 kB/s | 29 kB 00:00 2024-03-16T08:04:55.952 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T08:04:55.986 INFO:teuthology.orchestra.run.smithi129.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 21 MB/s | 1.2 MB 00:00 2024-03-16T08:04:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:04:56.070 INFO:teuthology.orchestra.run.smithi129.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 26 MB/s | 2.2 MB 00:00 2024-03-16T08:04:56.102 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:04:56.170 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:04:56.170 INFO:teuthology.orchestra.run.smithi129.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 23 MB/s | 2.3 MB 00:00 2024-03-16T08:04:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-03-16T08:04:56.245 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:04:56.262 INFO:teuthology.orchestra.run.smithi129.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 745 kB/s | 31 kB 00:00 2024-03-16T08:04:56.321 INFO:teuthology.orchestra.run.smithi129.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-16T08:04:56.396 INFO:teuthology.orchestra.run.smithi129.stdout:(172/174): librados2-19.0.0-2162.g3d78f5ba.el9. 1.5 MB/s | 3.4 MB 00:02 2024-03-16T08:04:56.457 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:04:56.488 INFO:teuthology.orchestra.run.smithi129.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 445 kB/s | 74 kB 00:00 2024-03-16T08:04:56.567 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T08:04:56.661 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T08:04:56.711 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T08:04:56.735 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T08:04:56.770 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T08:04:56.845 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T08:04:56.898 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T08:04:56.936 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T08:04:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T08:04:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T08:04:57.079 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T08:04:57.111 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T08:04:57.140 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T08:04:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T08:04:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T08:04:57.289 INFO:teuthology.orchestra.run.smithi129.stdout:(174/174): ceph-test-19.0.0-2162.g3d78f5ba.el9. 3.9 MB/s | 48 MB 00:12 2024-03-16T08:04:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:04:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:Total 15 MB/s | 224 MB 00:14 2024-03-16T08:04:57.296 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T08:04:57.325 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T08:04:57.393 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T08:04:57.439 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T08:04:57.498 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-16T08:04:57.512 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T08:04:57.585 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T08:04:57.620 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T08:04:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T08:04:57.666 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:04:57.677 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:04:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T08:04:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T08:04:57.768 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-03-16T08:04:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T08:04:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T08:04:57.853 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T08:04:57.907 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T08:04:58.038 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-03-16T08:04:58.038 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-03-16T08:04:58.223 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T08:04:58.254 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T08:04:58.304 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T08:04:58.323 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:04:58.372 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T08:04:58.415 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:04:58.423 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:04:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:04:58.455 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T08:04:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:04:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T08:04:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:04:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T08:04:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T08:04:58.572 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:04:58.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T08:04:58.789 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T08:04:58.809 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T08:04:58.860 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T08:04:58.908 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:04:58.933 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T08:04:59.015 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T08:04:59.028 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:04:59.056 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:04:59.059 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T08:04:59.121 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:04:59.160 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T08:04:59.182 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:04:59.190 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T08:04:59.199 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:04:59.237 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T08:04:59.292 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T08:04:59.348 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T08:04:59.379 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T08:04:59.391 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:04:59.416 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T08:04:59.422 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:04:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T08:04:59.470 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T08:04:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T08:04:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T08:04:59.584 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T08:04:59.632 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T08:04:59.718 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T08:04:59.739 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:04:59.747 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-03-16T08:04:59.747 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-03-16T08:04:59.773 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T08:04:59.799 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:04:59.906 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:04:59.958 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:05:00.021 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T08:05:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T08:05:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T08:05:00.176 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:05:00.203 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T08:05:00.244 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T08:05:00.269 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T08:05:00.284 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T08:05:00.285 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T08:05:00.360 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T08:05:00.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T08:05:00.422 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T08:05:00.458 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T08:05:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T08:05:00.502 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T08:05:00.579 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T08:05:00.583 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T08:05:00.626 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T08:05:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T08:05:00.675 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T08:05:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T08:05:00.739 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T08:05:00.752 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T08:05:00.792 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T08:05:00.835 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T08:05:00.869 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T08:05:00.895 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T08:05:00.929 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T08:05:00.968 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T08:05:00.972 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T08:05:01.000 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T08:05:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T08:05:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T08:05:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T08:05:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T08:05:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T08:05:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T08:05:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T08:05:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T08:05:01.236 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T08:05:01.262 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T08:05:01.282 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T08:05:01.301 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T08:05:01.332 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T08:05:01.371 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T08:05:01.427 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T08:05:01.447 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T08:05:01.482 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T08:05:01.491 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T08:05:01.523 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T08:05:01.564 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T08:05:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T08:05:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T08:05:01.778 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T08:05:01.830 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T08:05:01.862 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T08:05:01.917 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T08:05:02.015 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T08:05:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T08:05:02.072 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T08:05:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T08:05:02.224 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T08:05:02.279 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:05:02.289 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:05:02.330 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T08:05:02.377 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T08:05:02.415 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T08:05:02.445 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T08:05:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T08:05:02.805 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T08:05:02.852 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T08:05:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T08:05:02.904 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T08:05:02.934 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T08:05:03.002 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T08:05:03.021 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T08:05:03.041 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T08:05:03.061 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T08:05:03.179 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T08:05:03.227 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T08:05:03.234 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T08:05:03.267 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T08:05:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T08:05:03.346 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T08:05:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T08:05:03.388 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T08:05:03.425 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T08:05:03.472 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T08:05:03.549 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T08:05:03.593 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T08:05:03.644 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T08:05:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T08:05:03.831 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T08:05:03.861 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T08:05:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T08:05:03.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T08:05:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T08:05:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T08:05:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T08:05:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T08:05:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T08:05:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T08:05:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T08:05:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T08:05:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T08:05:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T08:05:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T08:05:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T08:05:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T08:05:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T08:05:04.994 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T08:05:05.055 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T08:05:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T08:05:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T08:05:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T08:05:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T08:05:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T08:05:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T08:05:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T08:05:05.330 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T08:05:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T08:05:05.401 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T08:05:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T08:05:05.432 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T08:05:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T08:05:05.463 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T08:05:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T08:05:05.490 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T08:05:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T08:05:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T08:05:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T08:05:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T08:05:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T08:05:05.806 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T08:05:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T08:05:05.829 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:05.918 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T08:05:05.950 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T08:05:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T08:05:05.986 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T08:05:06.440 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T08:05:06.465 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:06.536 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:06.558 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T08:05:06.592 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-03-16T08:05:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T08:05:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T08:05:06.763 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T08:05:06.782 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:05:06.791 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T08:05:06.831 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T08:05:06.885 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T08:05:06.919 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T08:05:06.935 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T08:05:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T08:05:06.968 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T08:05:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T08:05:07.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T08:05:07.120 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T08:05:07.169 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T08:05:07.306 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T08:05:07.354 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T08:05:07.354 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:05:07.383 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T08:05:07.460 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:05:07.468 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T08:05:07.488 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:05:07.581 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:05:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:05:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T08:05:07.824 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T08:05:07.873 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T08:05:07.901 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:05:08.006 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:05:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:05:08.091 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T08:05:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T08:05:08.163 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T08:05:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:05:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T08:05:08.285 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T08:05:08.370 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:05:08.401 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T08:05:08.441 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T08:05:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T08:05:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:05:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T08:05:08.772 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:08.776 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:05:08.813 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:08.841 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:05:09.042 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T08:05:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T08:05:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T08:05:09.279 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T08:05:09.323 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T08:05:09.356 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:09.358 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T08:05:09.441 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T08:05:09.441 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:09.494 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T08:05:09.522 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:09.522 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T08:05:09.522 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T08:05:09.523 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:09.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T08:05:09.550 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T08:05:09.607 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T08:05:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T08:05:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T08:05:09.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T08:05:09.708 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T08:05:09.724 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T08:05:09.736 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T08:05:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T08:05:09.790 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T08:05:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T08:05:09.841 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T08:05:09.876 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T08:05:09.905 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T08:05:09.973 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T08:05:10.019 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T08:05:10.093 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T08:05:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T08:05:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:10.226 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T08:05:10.245 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:10.275 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T08:05:10.280 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T08:05:10.312 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T08:05:10.795 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T08:05:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T08:05:10.839 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:10.874 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T08:05:10.911 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:10.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T08:05:10.947 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T08:05:11.054 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:05:11.062 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T08:05:11.288 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T08:05:11.328 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T08:05:11.366 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T08:05:11.395 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T08:05:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T08:05:11.810 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T08:05:11.873 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T08:05:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T08:05:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T08:05:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T08:05:12.149 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T08:05:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T08:05:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T08:05:12.274 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T08:05:12.320 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T08:05:12.356 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T08:05:12.397 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T08:05:12.439 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T08:05:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T08:05:12.612 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T08:05:12.642 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T08:05:12.684 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T08:05:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T08:05:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T08:05:12.866 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T08:05:12.892 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T08:05:12.917 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T08:05:12.989 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T08:05:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T08:05:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T08:05:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T08:05:13.297 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:13.307 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T08:05:13.331 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:13.362 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T08:05:13.555 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T08:05:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T08:05:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T08:05:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T08:05:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T08:05:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T08:05:13.861 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T08:05:13.896 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T08:05:13.902 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:13.942 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:13.957 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T08:05:13.991 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T08:05:14.022 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T08:05:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:14.034 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T08:05:14.034 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T08:05:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:14.106 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T08:05:14.140 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T08:05:14.173 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T08:05:14.215 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T08:05:14.250 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T08:05:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T08:05:14.428 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T08:05:14.472 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T08:05:14.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T08:05:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T08:05:14.648 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T08:05:14.706 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T08:05:15.429 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T08:05:15.478 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T08:05:15.512 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T08:05:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T08:05:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T08:05:15.680 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T08:05:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T08:05:15.886 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T08:05:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T08:05:16.588 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T08:05:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T08:05:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T08:05:18.014 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T08:05:18.045 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T08:05:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T08:05:18.103 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T08:05:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T08:05:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:18.532 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T08:05:18.564 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T08:05:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T08:05:19.059 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T08:05:19.096 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T08:05:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T08:05:21.265 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:21.265 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-03-16T08:05:21.265 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-03-16T08:05:21.265 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-03-16T08:05:21.266 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-03-16T08:05:21.266 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-03-16T08:05:21.266 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-03-16T08:05:21.266 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-03-16T08:05:21.266 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:21.429 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:21.445 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:21.445 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T08:05:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:21.477 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:22.012 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T08:05:22.048 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:22.129 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T08:05:22.129 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T08:05:22.129 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T08:05:22.130 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:22.157 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:22.966 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:22.967 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T08:05:22.967 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:22.967 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:23.191 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:23.213 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:23.214 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:23.214 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T08:05:23.214 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:23.214 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:23.214 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:23.667 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:23.690 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:23.794 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T08:05:24.203 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:24.233 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:24.266 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T08:05:24.326 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T08:05:24.357 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:24.592 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:24.624 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:25.590 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:25.625 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:25.725 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:25.879 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T08:05:25.901 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:25.924 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:25.924 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:25.925 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T08:05:25.925 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:25.925 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:25.925 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:25.969 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:25.979 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:26.026 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-03-16T08:05:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:26.430 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:26.430 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T08:05:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:26.898 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T08:05:26.929 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:26.951 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:26.952 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:26.952 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T08:05:26.952 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:26.952 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:26.952 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:26.996 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:27.018 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:27.018 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:27.018 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T08:05:27.018 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:27.086 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:27.086 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:27.086 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T08:05:27.086 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:27.086 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:27.087 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:27.400 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:27.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:27.985 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:27.986 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:27.986 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T08:05:27.986 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:27.986 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:27.986 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:28.241 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:28.701 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:28.725 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T08:05:29.281 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:29.309 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:29.334 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T08:05:29.388 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T08:05:29.416 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:29.664 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:29.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:30.709 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:30.734 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:30.841 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:31.012 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T08:05:31.044 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:31.068 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T08:05:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:31.068 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:31.069 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:31.161 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:31.173 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:31.216 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T08:05:31.888 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:31.955 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T08:05:31.984 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T08:05:32.011 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T08:05:32.130 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T08:05:32.163 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:32.187 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:32.187 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:32.187 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T08:05:32.188 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:32.188 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:32.188 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:32.193 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T08:05:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:32.253 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:32.253 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T08:05:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:32.306 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:32.310 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:32.330 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:32.350 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T08:05:32.390 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T08:05:32.442 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T08:05:32.486 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T08:05:32.519 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T08:05:32.558 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T08:05:32.588 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T08:05:32.634 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T08:05:32.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:32.689 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:32.689 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:32.689 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T08:05:32.689 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:32.690 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:32.690 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:32.695 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T08:05:32.720 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T08:05:33.333 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:33.368 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:33.388 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-03-16T08:05:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T08:05:34.084 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:34.084 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T08:05:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:34.216 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T08:05:34.304 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T08:05:34.346 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T08:05:34.427 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T08:05:34.466 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T08:05:34.491 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T08:05:34.516 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T08:05:34.543 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T08:05:34.543 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:34.631 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:34.669 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T08:05:34.669 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:34.754 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:34.780 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T08:05:34.813 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T08:05:34.838 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T08:05:34.863 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T08:05:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T08:05:34.936 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T08:05:34.968 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T08:05:34.993 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T08:05:35.019 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T08:05:35.044 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:35.069 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T08:05:35.075 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:35.076 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:35.263 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:35.264 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T08:05:35.429 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T08:05:35.456 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T08:05:35.520 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:05:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:35.813 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:05:35.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:36.290 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:05:36.313 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:36.409 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T08:05:36.851 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:36.887 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T08:05:36.923 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T08:05:36.983 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T08:05:37.006 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:37.242 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T08:05:37.274 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T08:05:37.482 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T08:05:37.501 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T08:05:37.687 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T08:05:37.805 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T08:05:37.850 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T08:05:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T08:05:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T08:05:38.010 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T08:05:38.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T08:05:38.092 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T08:05:38.121 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T08:05:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T08:05:38.242 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T08:05:38.244 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T08:05:38.274 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:38.277 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T08:05:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T08:05:38.546 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T08:05:38.568 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:05:38.592 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:38.669 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:38.680 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T08:05:38.725 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T08:05:38.918 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:38.958 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:38.979 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T08:05:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T08:05:39.635 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:39.635 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T08:05:39.635 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:39.636 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:05:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:39.689 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:39.711 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T08:05:39.711 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:39.711 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T08:05:39.711 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T08:05:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T08:05:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T08:05:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T08:05:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T08:05:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T08:05:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T08:05:40.113 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:05:40.113 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T08:05:40.113 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:40.114 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:05:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T08:05:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T08:05:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T08:05:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T08:05:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T08:05:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T08:05:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T08:05:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T08:05:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T08:05:40.541 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:05:40.542 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T08:05:40.543 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T08:05:40.544 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T08:05:40.544 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T08:05:40.544 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T08:05:40.559 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T08:05:40.560 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T08:05:40.561 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T08:05:40.562 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T08:05:40.563 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T08:05:40.564 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T08:05:40.565 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T08:05:40.566 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T08:05:40.567 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T08:05:40.568 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T08:05:40.569 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T08:05:40.570 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T08:05:40.571 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T08:05:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T08:05:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T08:05:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T08:05:40.668 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T08:05:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T08:05:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:40.905 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:40.905 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:40.987 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T08:05:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T08:05:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T08:05:41.165 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:43.418 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T08:05:43.418 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:43.418 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-03-16T08:05:43.418 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T08:05:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-16.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T08:05:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T08:05:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T08:05:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T08:05:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T08:05:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T08:05:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T08:05:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:05:43.431 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-03-16T08:05:43.896 DEBUG:teuthology.parallel:result is None 2024-03-16T08:05:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T08:05:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T08:05:44.636 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T08:05:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T08:05:44.935 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T08:05:44.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T08:05:45.023 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T08:05:45.077 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T08:05:45.121 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T08:05:45.154 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T08:05:45.186 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T08:05:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T08:05:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T08:05:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T08:05:45.356 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T08:05:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:45.940 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:45.940 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T08:05:45.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T08:05:45.942 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T08:05:45.943 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T08:05:45.944 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T08:05:45.945 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T08:05:45.946 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T08:05:45.947 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T08:05:45.948 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T08:05:45.949 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T08:05:45.950 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T08:05:45.951 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T08:05:45.952 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T08:05:45.953 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T08:05:45.954 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T08:05:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:45.991 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T08:05:46.814 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T08:05:46.901 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T08:05:46.941 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T08:05:47.025 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T08:05:47.055 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T08:05:47.079 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T08:05:47.105 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T08:05:47.132 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T08:05:47.132 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:47.219 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T08:05:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T08:05:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:47.334 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T08:05:47.360 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T08:05:47.393 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T08:05:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T08:05:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T08:05:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T08:05:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T08:05:47.549 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T08:05:47.574 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T08:05:47.600 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T08:05:47.625 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:47.650 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T08:05:47.657 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:47.657 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:47.880 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T08:05:47.881 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:47.970 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T08:05:47.994 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:48.020 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T08:05:48.053 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T08:05:48.079 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T08:05:48.152 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:48.829 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:48.830 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.831 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.832 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-03-16T08:05:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T08:05:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T08:05:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T08:05:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T08:05:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T08:05:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T08:05:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T08:05:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T08:05:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:05:48.842 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-16T08:05:49.367 DEBUG:teuthology.parallel:result is None 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T08:05:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T08:05:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T08:05:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T08:05:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T08:05:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T08:05:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T08:05:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T08:05:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T08:05:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T08:05:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T08:05:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T08:05:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T08:05:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T08:05:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T08:05:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T08:05:55.719 INFO:teuthology.orchestra.run.smithi129.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:55.720 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.721 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.722 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: jq-1.6-16.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T08:05:55.723 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T08:05:55.724 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T08:05:55.725 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T08:05:55.726 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T08:05:55.727 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T08:05:55.728 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T08:05:55.729 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T08:05:55.730 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T08:05:55.731 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T08:05:55.731 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-03-16T08:05:55.731 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-03-16T08:05:56.224 DEBUG:teuthology.parallel:result is None 2024-03-16T08:05:56.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:56.445 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T08:05:56.473 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.473 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.473 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T08:05:56.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:56.620 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T08:05:56.641 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.641 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.641 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T08:05:56.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:56.780 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T08:05:56.806 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.806 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T08:05:56.806 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T08:05:56.808 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T08:05:56.808 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:05:56.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T08:05:56.848 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:05:56.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T08:05:56.878 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:05:56.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T08:05:56.912 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T08:05:56.912 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:05:56.912 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T08:05:56.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T08:05:57.021 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:05:57.021 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T08:05:57.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T08:05:57.114 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:05:57.114 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T08:05:57.146 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T08:05:57.218 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T08:05:57.218 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:05:57.218 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T08:05:57.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T08:05:57.326 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:05:57.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T08:05:57.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T08:05:57.419 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:05:57.419 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T08:05:57.446 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T08:05:57.513 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T08:05:57.514 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:05:57.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T08:05:57.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T08:05:57.624 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:05:57.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T08:05:57.650 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T08:05:57.717 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:05:57.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T08:05:57.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T08:05:57.811 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-16T08:05:57.905 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-16T08:05:57.905 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:57.906 INFO:tasks.cephadm:Cluster fsid is 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:05:57.906 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-16T08:05:57.906 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.b': '172.21.15.129', 'mon.c': '172.21.15.154'} 2024-03-16T08:05:57.906 INFO:tasks.cephadm:First mon is mon.a on smithi049 2024-03-16T08:05:57.906 INFO:tasks.cephadm:First mgr is x 2024-03-16T08:05:57.906 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-16T08:05:57.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-03-16T08:05:57.943 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-03-16T08:05:57.970 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-03-16T08:05:57.997 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-16T08:05:57.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:58.142 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77658/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2162-g3d78f5ba', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2162.g3d78f5ba'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 01:33:04.210510', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240314.183243', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-16T08:05:58.281 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20240314.183243, sha1 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:05:58.282 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm 2024-03-16T08:05:58.283 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:58.879 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 08:05 /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:58.879 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.345 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 08:05 /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.345 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.947 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 08:05 /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.947 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.967 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T08:05:59.985 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T08:06:00.020 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af on all hosts... 2024-03-16T08:06:00.020 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T08:06:00.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T08:06:00.028 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T08:06:00.870 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T08:06:00.878 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T08:06:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:06:29.345 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:06:54.573 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-16T08:06:54.573 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T08:06:54.573 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T08:06:54.573 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-03-16T08:06:54.573 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T08:06:54.574 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-03-16T08:06:54.574 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-03-16T08:06:58.721 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-03-16T08:06:58.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-03-16T08:06:58.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-03-16T08:06:58.825 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-03-16T08:06:58.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-03-16T08:06:58.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-03-16T08:06:58.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-03-16T08:06:58.955 INFO:tasks.cephadm:Writing seed config... 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-16T08:06:58.956 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-16T08:06:58.957 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-16T08:06:58.958 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-16T08:06:58.959 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-16T08:06:58.959 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-16T08:06:58.959 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-16T08:06:58.959 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-16T08:06:58.959 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-16T08:06:58.959 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:06:58.960 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-16T08:06:58.978 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 053f2308-e36c-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-16T08:06:58.979 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service 2024-03-16T08:06:59.021 DEBUG:teuthology.orchestra.run.smithi049:mgr.x> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x.service 2024-03-16T08:06:59.064 INFO:tasks.cephadm:Bootstrapping... 2024-03-16T08:06:59.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -v bootstrap --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:06:59.269 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:06:59.269 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', '-v', 'bootstrap', '--fsid', '053f2308-e36c-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-03-16T08:06:59.296 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:06:59.296 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-16T08:06:59.296 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-03-16T08:06:59.322 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:06:59.322 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-03-16T08:06:59.322 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-03-16T08:06:59.330 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T08:06:59.330 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T08:06:59.336 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T08:06:59.336 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:06:59.343 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-16T08:06:59.350 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-16T08:06:59.350 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-16T08:06:59.350 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-03-16T08:06:59.374 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:06:59.375 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-16T08:06:59.375 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-03-16T08:06:59.375 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-03-16T08:06:59.381 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T08:06:59.381 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T08:06:59.388 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T08:06:59.388 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:06:59.395 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 140197677473216 on /run/cephadm/053f2308-e36c-11ee-95c9-87774f69a715.lock 2024-03-16T08:06:59.401 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 140197677473216 acquired on /run/cephadm/053f2308-e36c-11ee-95c9-87774f69a715.lock 2024-03-16T08:06:59.402 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-03-16T08:06:59.402 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-03-16T08:06:59.402 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-16T08:06:59.406 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-03-16T08:06:59.406 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-03-16T08:06:59.409 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-16T08:06:59.409 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-16T08:06:59.409 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T08:06:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-16T08:06:59.413 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-16T08:06:59.413 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-16T08:06:59.413 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-16T08:06:59.414 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:66487374cb611c95435850c17b53ebb64c22ed9555f72cd43d678b60262a12b1 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T08:07:00.960 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-16T08:07:01.846 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T08:07:01.846 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T08:07:01.847 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-03-16T08:07:02.785 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-03-16T08:07:02.785 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-03-16T08:07:03.683 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAnU/VlS7K+BxAAhHYeHx1dtrjye7N1285T9g== 2024-03-16T08:07:04.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAoU/VlK0UsARAA3+qRwQf5nWnrSGIvYLjxag== 2024-03-16T08:07:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAoU/Vl9aoyORAA2fApBOgF2YsSKkPtKDrA1w== 2024-03-16T08:07:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T08:07:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:07:06.510 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-03-16T08:07:07.465 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2024-03-16T08:07:07.779 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-16T08:07:07.920 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-16T08:07:08.096 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target. 2024-03-16T08:07:08.097 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target. 2024-03-16T08:07:08.340 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a 2024-03-16T08:07:08.340 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-16T08:07:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target.wants/ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715@.service. 2024-03-16T08:07:08.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:08 smithi049 podman[31551]: 2024-03-16 08:07:08.651211278 +0000 UTC m=+0.019117888 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:07:09.073 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:07:09.073 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-16T08:07:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:07:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:07:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-16T08:07:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T08:07:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-03-16T08:07:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:08 smithi049 podman[31551]: 2024-03-16 08:07:08.977762697 +0000 UTC m=+0.345669296 container init 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:08 smithi049 podman[31551]: 2024-03-16 08:07:08.980732775 +0000 UTC m=+0.348639374 container start 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:09 smithi049 ceph-mon[31570]: mkfs 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:09 smithi049 ceph-mon[31570]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:09 smithi049 bash[31551]: 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee 2024-03-16T08:07:09.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:09 smithi049 systemd[1]: Started Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:07:10.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-03-16T08:07:10.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:10.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.555944s) 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.515+0000 7f1c96d67700 1 Processor -- start 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.515+0000 7f1c96d67700 1 -- start start 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.515+0000 7f1c96d67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.515+0000 7f1c96d67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901086c0 con 0x7f1c90107cd0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c94b03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c94b03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46890/0 (socket says 172.21.15.49:46890) 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c94b03700 1 -- 172.21.15.49:0/3552710955 learned_addr learned my addr 172.21.15.49:0/3552710955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c94b03700 1 -- 172.21.15.49:0/3552710955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c90108e80 con 0x7f1c90107cd0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c94b03700 1 --2- 172.21.15.49:0/3552710955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e80 tx=0x7f1c8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3f0c4aa1845fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c8f7fe700 1 -- 172.21.15.49:0/3552710955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c88017070 con 0x7f1c90107cd0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c8f7fe700 1 -- 172.21.15.49:0/3552710955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c88003a20 con 0x7f1c90107cd0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.516+0000 7f1c8f7fe700 1 -- 172.21.15.49:0/3552710955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c88003d20 con 0x7f1c90107cd0 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 -- 172.21.15.49:0/3552710955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 msgr2=0x7f1c901080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 --2- 172.21.15.49:0/3552710955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e80 tx=0x7f1c8800b330 comp rx=0 tx=0).stop 2024-03-16T08:07:10.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 -- 172.21.15.49:0/3552710955 shutdown_connections 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 --2- 172.21.15.49:0/3552710955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c901080f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 -- 172.21.15.49:0/3552710955 >> 172.21.15.49:0/3552710955 conn(0x7f1c901032b0 msgr2=0x7f1c901056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 -- 172.21.15.49:0/3552710955 shutdown_connections 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 -- 172.21.15.49:0/3552710955 wait complete. 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.517+0000 7f1c96d67700 1 Processor -- start 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c96d67700 1 -- start start 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c96d67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c96d67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901093b0 con 0x7f1c90107cd0 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c94b03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c94b03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46900/0 (socket says 172.21.15.49:46900) 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c94b03700 1 -- 172.21.15.49:0/2663147487 learned_addr learned my addr 172.21.15.49:0/2663147487 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c94b03700 1 -- 172.21.15.49:0/2663147487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9019b890 con 0x7f1c90107cd0 2024-03-16T08:07:10.152 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c94b03700 1 --2- 172.21.15.49:0/2663147487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c88004000 tx=0x7f1c88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c88022020 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c88018040 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.518+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9019bb20 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.519+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c88017070 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.519+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9019e810 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.519+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c88025020 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.519+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c8800f050 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.520+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c900693a0 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.522+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1c8802c030 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.598+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1c9002dd00 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.598+0000 7f1c8dffb700 1 -- 172.21.15.49:0/2663147487 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1c8801e480 con 0x7f1c90107cd0 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 msgr2=0x7f1c9019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 --2- 172.21.15.49:0/2663147487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c88004000 tx=0x7f1c88004ad0 comp rx=0 tx=0).stop 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 shutdown_connections 2024-03-16T08:07:10.153 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 --2- 172.21.15.49:0/2663147487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c90107cd0 0x7f1c9019b350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:10.154 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 >> 172.21.15.49:0/2663147487 conn(0x7f1c901032b0 msgr2=0x7f1c90191fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:10.154 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 shutdown_connections 2024-03-16T08:07:10.154 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:09.599+0000 7f1c96d67700 1 -- 172.21.15.49:0/2663147487 wait complete. 2024-03-16T08:07:10.154 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-03-16T08:07:10.154 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-03-16T08:07:10.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T08:07:10.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: monmap epoch 1 2024-03-16T08:07:10.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:10.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: last_changed 2024-03-16T08:07:05.779270+0000 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: min_mon_release 19 (squid) 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: election_strategy: 1 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: fsmap 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: mgrmap e1: no daemons active 2024-03-16T08:07:10.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:10 smithi049 ceph-mon[31570]: from='client.? 172.21.15.49:0/2663147487' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-16T08:07:11.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:11.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T08:07:11.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T08:07:11.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.530+0000 7f140c375700 1 Processor -- start 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.530+0000 7f140c375700 1 -- start start 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.530+0000 7f140c375700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.530+0000 7f140c375700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f140407bd80 con 0x7f14041087f0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.531+0000 7f140a111700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.531+0000 7f140a111700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46908/0 (socket says 172.21.15.49:46908) 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.531+0000 7f140a111700 1 -- 172.21.15.49:0/3203884427 learned_addr learned my addr 172.21.15.49:0/3203884427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.531+0000 7f140a111700 1 -- 172.21.15.49:0/3203884427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140410b120 con 0x7f14041087f0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.531+0000 7f140a111700 1 --2- 172.21.15.49:0/3203884427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f14000098a0 tx=0x7f1400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1e8a5915a193fc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140910f700 1 -- 172.21.15.49:0/3203884427 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1400004160 con 0x7f14041087f0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140910f700 1 -- 172.21.15.49:0/3203884427 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400004300 con 0x7f14041087f0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140910f700 1 -- 172.21.15.49:0/3203884427 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14000036a0 con 0x7f14041087f0 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 -- 172.21.15.49:0/3203884427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 msgr2=0x7f140410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 --2- 172.21.15.49:0/3203884427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f14000098a0 tx=0x7f1400009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 -- 172.21.15.49:0/3203884427 shutdown_connections 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 --2- 172.21.15.49:0/3203884427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f140410abe0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:11.149 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 -- 172.21.15.49:0/3203884427 >> 172.21.15.49:0/3203884427 conn(0x7f1404102160 msgr2=0x7f14041045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 -- 172.21.15.49:0/3203884427 shutdown_connections 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.532+0000 7f140c375700 1 -- 172.21.15.49:0/3203884427 wait complete. 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140c375700 1 Processor -- start 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140c375700 1 -- start start 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140c375700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140c375700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f140410b530 con 0x7f14041087f0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140a111700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140a111700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46914/0 (socket says 172.21.15.49:46914) 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140a111700 1 -- 172.21.15.49:0/3065158979 learned_addr learned my addr 172.21.15.49:0/3065158979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.533+0000 7f140a111700 1 -- 172.21.15.49:0/3065158979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041085b0 con 0x7f14041087f0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f140a111700 1 --2- 172.21.15.49:0/3065158979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1400000c00 tx=0x7f1400004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1400003db0 con 0x7f14041087f0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400011630 con 0x7f14041087f0 2024-03-16T08:07:11.150 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1400011950 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041067a0 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1404106ce0 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1400011af0 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.534+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f140001c450 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.535+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14040693a0 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.537+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1400004260 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.608+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1404194a60 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.611+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f1404194a60 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.611+0000 7f13fb7fe700 1 -- 172.21.15.49:0/3065158979 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f140001cbb0 con 0x7f14041087f0 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 msgr2=0x7f1404108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 --2- 172.21.15.49:0/3065158979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1400000c00 tx=0x7f1400004750 comp rx=0 tx=0).stop 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 shutdown_connections 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 --2- 172.21.15.49:0/3065158979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14041087f0 0x7f1404108070 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 >> 172.21.15.49:0/3065158979 conn(0x7f1404102160 msgr2=0x7f14041044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 shutdown_connections 2024-03-16T08:07:11.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:10.612+0000 7f140c375700 1 -- 172.21.15.49:0/3065158979 wait complete. 2024-03-16T08:07:11.152 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-03-16T08:07:11.265 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:11 smithi049 ceph-mon[31570]: from='client.? 172.21.15.49:0/3065158979' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T08:07:11.265 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:11 smithi049 ceph-mon[31570]: from='client.? 172.21.15.49:0/3065158979' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-16T08:07:12.195 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 ceph-mon[31570]: from='client.? 172.21.15.49:0/2515372047' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.555+0000 7f132a581700 1 Processor -- start 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f132a581700 1 -- start start 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f132a581700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f132a581700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1324106990 con 0x7f1324105fa0 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f1323fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f1323fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46924/0 (socket says 172.21.15.49:46924) 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f1323fff700 1 -- 172.21.15.49:0/818779666 learned_addr learned my addr 172.21.15.49:0/818779666 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f1323fff700 1 -- 172.21.15.49:0/818779666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1324108f30 con 0x7f1324105fa0 2024-03-16T08:07:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.556+0000 7f1323fff700 1 --2- 172.21.15.49:0/818779666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f13140098a0 tx=0x7f1314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c6cb842f9db5f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f1322ffd700 1 -- 172.21.15.49:0/818779666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314004160 con 0x7f1324105fa0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f1322ffd700 1 -- 172.21.15.49:0/818779666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1314004300 con 0x7f1324105fa0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f1322ffd700 1 -- 172.21.15.49:0/818779666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314011b50 con 0x7f1324105fa0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 -- 172.21.15.49:0/818779666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 msgr2=0x7f13241063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 --2- 172.21.15.49:0/818779666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f13140098a0 tx=0x7f1314009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 -- 172.21.15.49:0/818779666 shutdown_connections 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 --2- 172.21.15.49:0/818779666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241063c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 -- 172.21.15.49:0/818779666 >> 172.21.15.49:0/818779666 conn(0x7f1324101910 msgr2=0x7f1324103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 -- 172.21.15.49:0/818779666 shutdown_connections 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.557+0000 7f132a581700 1 -- 172.21.15.49:0/818779666 wait complete. 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.558+0000 7f132a581700 1 Processor -- start 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.558+0000 7f132a581700 1 -- start start 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.558+0000 7f132a581700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.558+0000 7f132a581700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f132410c2e0 con 0x7f1324105fa0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.558+0000 7f1323fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:12.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f1323fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46940/0 (socket says 172.21.15.49:46940) 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f1323fff700 1 -- 172.21.15.49:0/2515372047 learned_addr learned my addr 172.21.15.49:0/2515372047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f1323fff700 1 -- 172.21.15.49:0/2515372047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1324108cf0 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f1323fff700 1 --2- 172.21.15.49:0/2515372047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314003b20 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1314003cc0 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1324106ee0 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314024910 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1324107420 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1314022020 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.559+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1314011e00 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.560+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13240693a0 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.563+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f131401acd0 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.633+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1324107950 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.634+0000 7f13217fa700 1 -- 172.21.15.49:0/2515372047 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1314032030 con 0x7f1324105fa0 2024-03-16T08:07:12.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.634+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 msgr2=0x7f13241087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.634+0000 7f132a581700 1 --2- 172.21.15.49:0/2515372047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.635+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 shutdown_connections 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.635+0000 7f132a581700 1 --2- 172.21.15.49:0/2515372047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1324105fa0 0x7f13241087b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.635+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 >> 172.21.15.49:0/2515372047 conn(0x7f1324101910 msgr2=0x7f1324102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.635+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 shutdown_connections 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:11.635+0000 7f132a581700 1 -- 172.21.15.49:0/2515372047 wait complete. 2024-03-16T08:07:12.205 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-03-16T08:07:12.482 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 systemd[1]: Stopping Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:07:12.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a[31566]: 2024-03-16T08:07:12.481+0000 7fb2b4342700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:07:12.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a[31566]: 2024-03-16T08:07:12.481+0000 7fb2b4342700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-16T08:07:12.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 podman[31816]: 2024-03-16 08:07:12.606148527 +0000 UTC m=+0.230890179 container died 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T08:07:13.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 podman[31816]: 2024-03-16 08:07:12.957996047 +0000 UTC m=+0.582737705 container cleanup 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:07:13.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:12 smithi049 bash[31816]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a 2024-03-16T08:07:13.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 podman[31830]: 2024-03-16 08:07:13.217113312 +0000 UTC m=+0.609961908 container remove 5daaf4809223bbf53807d8b3880d3beafd4824e8353c1d6ac4c40e320b2af2ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T08:07:13.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-16T08:07:13.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 systemd[1]: Stopped Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:07:13.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 systemd[1]: Starting Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:07:13.840 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-16T08:07:13.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 podman[31923]: 2024-03-16 08:07:13.496912905 +0000 UTC m=+0.018359964 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:07:13.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 podman[31923]: 2024-03-16 08:07:13.601324709 +0000 UTC m=+0.122771755 container create 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T08:07:13.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 podman[31923]: 2024-03-16 08:07:13.77062362 +0000 UTC m=+0.292070679 container init 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-16T08:07:13.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 podman[31923]: 2024-03-16 08:07:13.773598549 +0000 UTC m=+0.295045596 container start 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-16T08:07:13.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: pidfile_write: ignore empty --pid-file 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: load: jerasure load: lrc 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: RocksDB version: 7.9.2 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Git sha 0 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: DB SUMMARY 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: DB Session ID: PVFJR8J29EFYBLR71K22 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: CURRENT file: CURRENT 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-16T08:07:13.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104752 ; 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.error_if_exists: 0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.create_if_missing: 0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.paranoid_checks: 1 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.env: 0x561bb9258c60 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.info_log: 0x561bba745ac0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.statistics: (nil) 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.use_fsync: 0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_log_file_size: 0 2024-03-16T08:07:13.877 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_fallocate: 1 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.use_direct_reads: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.db_log_dir: 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.wal_dir: 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T08:07:13.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.write_buffer_manager: 0x561bba7181e0 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T08:07:13.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.unordered_write: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.row_cache: None 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.wal_filter: None 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T08:07:13.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.two_write_queues: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.wal_compression: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.atomic_flush: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.log_readahead_size: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T08:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_background_jobs: 2 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_background_compactions: -1 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_subcompactions: 1 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_open_files: -1 2024-03-16T08:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_background_flushes: -1 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Compression algorithms supported: 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kZSTD supported: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kXpressCompression supported: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kZlibCompression supported: 1 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kSnappyCompression supported: 1 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kLZ4Compression supported: 1 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: kBZip2Compression supported: 0 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T08:07:13.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.merge_operator: 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_filter: None 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561bba745bc0) 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T08:07:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x561bba74b090 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-03-16T08:07:13.886 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-03-16T08:07:13.887 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression: NoCompression 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.num_levels: 7 2024-03-16T08:07:13.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T08:07:13.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T08:07:13.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T08:07:13.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.table_properties_collectors: 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.inplace_update_support: 0 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T08:07:13.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.bloom_locality: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.max_successive_merges: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.ttl: 2592000 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T08:07:13.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enable_blob_files: false 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.min_blob_size: 0 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1a4fd85d-a38c-4481-8e51-552339d42f3f 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576433804938, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-16T08:07:13.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576433805993, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99903, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97811, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14078, "raw_average_key_size": 49, "raw_value_size": 90169, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710576433, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1a4fd85d-a38c-4481-8e51-552339d42f3f", "db_session_id": "PVFJR8J29EFYBLR71K22", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576433806078, "job": 1, "event": "recovery_finished"} 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561bba83e000 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: DB pointer 0x561bba828000 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:07:13.895 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 99.40 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 99.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-16T08:07:13.896 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 31.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 31.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:07:13.897 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x561bba74b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???) e1 preinit fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).mds e1 new map 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).mds e1 print_map 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2024-03-16T08:07:13.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 bash[31923]: 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 systemd[1]: Started Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: monmap epoch 1 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: last_changed 2024-03-16T08:07:05.779270+0000 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: min_mon_release 19 (squid) 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: election_strategy: 1 2024-03-16T08:07:13.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:07:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: fsmap 2024-03-16T08:07:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T08:07:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:13 smithi049 ceph-mon[31942]: mgrmap e1: no daemons active 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.387+0000 7fb0281e8700 1 Processor -- start 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.387+0000 7fb0281e8700 1 -- start start 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.387+0000 7fb0281e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.387+0000 7fb0281e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb02007c860 con 0x7fb02007dde0 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb025f84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:15.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb025f84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46818/0 (socket says 172.21.15.49:46818) 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb025f84700 1 -- 172.21.15.49:0/4060829639 learned_addr learned my addr 172.21.15.49:0/4060829639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb025f84700 1 -- 172.21.15.49:0/4060829639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02007c9e0 con 0x7fb02007dde0 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb025f84700 1 --2- 172.21.15.49:0/4060829639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0098a0 tx=0x7fb01c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd80193353f5dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb024f82700 1 -- 172.21.15.49:0/4060829639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c004160 con 0x7fb02007dde0 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb024f82700 1 -- 172.21.15.49:0/4060829639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb01c004300 con 0x7fb02007dde0 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.388+0000 7fb024f82700 1 -- 172.21.15.49:0/4060829639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c011b50 con 0x7fb02007dde0 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 -- 172.21.15.49:0/4060829639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 msgr2=0x7fb02007c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 --2- 172.21.15.49:0/4060829639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0098a0 tx=0x7fb01c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 -- 172.21.15.49:0/4060829639 shutdown_connections 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 --2- 172.21.15.49:0/4060829639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02007c290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 -- 172.21.15.49:0/4060829639 >> 172.21.15.49:0/4060829639 conn(0x7fb020103270 msgr2=0x7fb0201056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 -- 172.21.15.49:0/4060829639 shutdown_connections 2024-03-16T08:07:15.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.389+0000 7fb0281e8700 1 -- 172.21.15.49:0/4060829639 wait complete. 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb0281e8700 1 Processor -- start 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb0281e8700 1 -- start start 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb0281e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb0281e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb02010e240 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb025f84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb025f84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46834/0 (socket says 172.21.15.49:46834) 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb025f84700 1 -- 172.21.15.49:0/1049946648 learned_addr learned my addr 172.21.15.49:0/1049946648 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb025f84700 1 -- 172.21.15.49:0/1049946648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02019e360 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.390+0000 7fb025f84700 1 --2- 172.21.15.49:0/1049946648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb01c011b60 tx=0x7fb01c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c003970 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb01c003b10 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02019e5f0 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01c024610 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a12e0 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb01c024ca0 con 0x7fb02007dde0 2024-03-16T08:07:15.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.391+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb01c011e00 con 0x7fb02007dde0 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.392+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020063900 con 0x7fb02007dde0 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.394+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb01c043dd0 con 0x7fb02007dde0 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.464+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb0201a14b0 con 0x7fb02007dde0 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb016ffd700 1 -- 172.21.15.49:0/1049946648 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb01c032030 con 0x7fb02007dde0 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 msgr2=0x7fb02019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb0281e8700 1 --2- 172.21.15.49:0/1049946648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb01c011b60 tx=0x7fb01c011c40 comp rx=0 tx=0).stop 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 shutdown_connections 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb0281e8700 1 --2- 172.21.15.49:0/1049946648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02007dde0 0x7fb02019de20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.466+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 >> 172.21.15.49:0/1049946648 conn(0x7fb020103270 msgr2=0x7fb020103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.467+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 shutdown_connections 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:14.467+0000 7fb0281e8700 1 -- 172.21.15.49:0/1049946648 wait complete. 2024-03-16T08:07:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-16T08:07:15.059 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:07:15.059 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-03-16T08:07:15.060 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-16T08:07:15.060 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-16T08:07:15.309 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x 2024-03-16T08:07:15.309 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-16T08:07:15.462 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715.target.wants/ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-053f2308-e36c-11ee-95c9-87774f69a715@.service. 2024-03-16T08:07:15.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:15 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1049946648' entity='client.admin' 2024-03-16T08:07:15.704 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 systemd[1]: Starting Ceph mgr.x for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:07:15.998 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 podman[32160]: 2024-03-16 08:07:15.70469032 +0000 UTC m=+0.113314317 container create 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T08:07:15.998 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 podman[32160]: 2024-03-16 08:07:15.609345313 +0000 UTC m=+0.017969318 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:07:15.998 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 podman[32160]: 2024-03-16 08:07:15.915966843 +0000 UTC m=+0.324590848 container init 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T08:07:15.998 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 podman[32160]: 2024-03-16 08:07:15.918546024 +0000 UTC m=+0.327170022 container start 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T08:07:15.998 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:15 smithi049 bash[32160]: 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c 2024-03-16T08:07:16.011 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:07:16.011 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-16T08:07:16.018 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:07:16.019 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:07:16.019 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-16T08:07:16.019 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T08:07:16.027 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:07:16.027 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-03-16T08:07:16.035 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-03-16T08:07:16.258 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:16 smithi049 systemd[1]: Started Ceph mgr.x for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:07:16.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:16.492+0000 7f670fb62200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:07:16.896 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:16 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1200164715' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:07:16.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:16.592+0000 7f670fb62200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:07:17.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:17.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:07:17.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:07:17.202 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:07:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:07:09.045494+0000", 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:17.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.463+0000 7f4bf3674700 1 Processor -- start 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.463+0000 7f4bf3674700 1 -- start start 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.463+0000 7f4bf3674700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.463+0000 7f4bf3674700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec07be40 con 0x7f4bec108750 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4bf1410700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4bf1410700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46854/0 (socket says 172.21.15.49:46854) 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4bf1410700 1 -- 172.21.15.49:0/1080366817 learned_addr learned my addr 172.21.15.49:0/1080366817 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4bf1410700 1 -- 172.21.15.49:0/1080366817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec10b080 con 0x7f4bec108750 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4bf1410700 1 --2- 172.21.15.49:0/1080366817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbf639f1552b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4be3fff700 1 -- 172.21.15.49:0/1080366817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc004160 con 0x7f4bec108750 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4be3fff700 1 -- 172.21.15.49:0/1080366817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bdc004300 con 0x7f4bec108750 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.464+0000 7f4be3fff700 1 -- 172.21.15.49:0/1080366817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc011b50 con 0x7f4bec108750 2024-03-16T08:07:17.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 -- 172.21.15.49:0/1080366817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 msgr2=0x7f4bec10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 --2- 172.21.15.49:0/1080366817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 -- 172.21.15.49:0/1080366817 shutdown_connections 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 --2- 172.21.15.49:0/1080366817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec10ab40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 -- 172.21.15.49:0/1080366817 >> 172.21.15.49:0/1080366817 conn(0x7f4bec1021a0 msgr2=0x7f4bec1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 -- 172.21.15.49:0/1080366817 shutdown_connections 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.465+0000 7f4bf3674700 1 -- 172.21.15.49:0/1080366817 wait complete. 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf3674700 1 Processor -- start 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf3674700 1 -- start start 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf3674700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf3674700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10c040 con 0x7f4bec108750 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf1410700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf1410700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46868/0 (socket says 172.21.15.49:46868) 2024-03-16T08:07:17.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf1410700 1 -- 172.21.15.49:0/1200164715 learned_addr learned my addr 172.21.15.49:0/1200164715 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf1410700 1 -- 172.21.15.49:0/1200164715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec108510 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.466+0000 7f4bf1410700 1 --2- 172.21.15.49:0/1200164715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc011b60 tx=0x7f4bdc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc003970 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bdc003b10 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec106700 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc024610 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec106c40 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4bdc024ca0 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.467+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4bdc011e00 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.468+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec0693a0 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.470+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4bdc01a9f0 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.546+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4bec107170 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.546+0000 7f4be27fc700 1 -- 172.21.15.49:0/1200164715 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4bdc01a430 con 0x7f4bec108750 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.547+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 msgr2=0x7f4bec107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:17.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.547+0000 7f4bf3674700 1 --2- 172.21.15.49:0/1200164715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc011b60 tx=0x7f4bdc011c40 comp rx=0 tx=0).stop 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.547+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 shutdown_connections 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.547+0000 7f4bf3674700 1 --2- 172.21.15.49:0/1200164715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec108750 0x7f4bec107fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.547+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 >> 172.21.15.49:0/1200164715 conn(0x7f4bec1021a0 msgr2=0x7f4bec102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.548+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 shutdown_connections 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:16.548+0000 7f4bf3674700 1 -- 172.21.15.49:0/1200164715 wait complete. 2024-03-16T08:07:17.208 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-03-16T08:07:17.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.271+0000 7f670fb62200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:07:17.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.367+0000 7f670fb62200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:07:17.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.511+0000 7f670fb62200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:07:17.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.610+0000 7f670fb62200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:07:17.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.695+0000 7f670fb62200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:07:18.087 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:17.973+0000 7f670fb62200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:07:18.087 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:18.087+0000 7f670fb62200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:07:18.455 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:18.176+0000 7f670fb62200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:07:18.455 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:18.274+0000 7f670fb62200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:07:18.455 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:18.454+0000 7f670fb62200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:07:19.574 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:19.488+0000 7f670fb62200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:07:19.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:19 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1431418878' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:07:19.989 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:19.638+0000 7f670fb62200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:07:19.989 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:19.988+0000 7f670fb62200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:07:20.269 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:20.083+0000 7f670fb62200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:07:20.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:07:20.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:07:20.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:07:20.283 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:07:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:07:09.045494+0000", 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.584+0000 7ff05f5ca700 1 Processor -- start 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.584+0000 7ff05f5ca700 1 -- start start 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.584+0000 7ff05f5ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.584+0000 7ff05f5ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff058108760 con 0x7ff058107d70 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff05d366700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff05d366700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46872/0 (socket says 172.21.15.49:46872) 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff05d366700 1 -- 172.21.15.49:0/1017623429 learned_addr learned my addr 172.21.15.49:0/1017623429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff05d366700 1 -- 172.21.15.49:0/1017623429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff058108f20 con 0x7ff058107d70 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff05d366700 1 --2- 172.21.15.49:0/1017623429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0540098a0 tx=0x7ff054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e837b1191fbcbc8f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff04ffff700 1 -- 172.21.15.49:0/1017623429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054004160 con 0x7ff058107d70 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff04ffff700 1 -- 172.21.15.49:0/1017623429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff054004300 con 0x7ff058107d70 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.585+0000 7ff04ffff700 1 -- 172.21.15.49:0/1017623429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054011b50 con 0x7ff058107d70 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1017623429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 msgr2=0x7ff058108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 --2- 172.21.15.49:0/1017623429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0540098a0 tx=0x7ff054009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:20.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1017623429 shutdown_connections 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 --2- 172.21.15.49:0/1017623429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff058108190 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1017623429 >> 172.21.15.49:0/1017623429 conn(0x7ff058103350 msgr2=0x7ff058105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1017623429 shutdown_connections 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.586+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1017623429 wait complete. 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05f5ca700 1 Processor -- start 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05f5ca700 1 -- start start 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05f5ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05f5ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810c150 con 0x7ff058107d70 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05d366700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05d366700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46884/0 (socket says 172.21.15.49:46884) 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05d366700 1 -- 172.21.15.49:0/1431418878 learned_addr learned my addr 172.21.15.49:0/1431418878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05d366700 1 -- 172.21.15.49:0/1431418878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05807e040 con 0x7ff058107d70 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.587+0000 7ff05d366700 1 --2- 172.21.15.49:0/1431418878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff054011850 tx=0x7ff054011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054003af0 con 0x7ff058107d70 2024-03-16T08:07:20.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05807c230 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff054003c90 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0540236b0 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05807c770 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff054022030 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.588+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff054011e00 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.589+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0580693a0 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.591+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff05401a980 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.675+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff05807c940 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.676+0000 7ff04e7fc700 1 -- 172.21.15.49:0/1431418878 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff0540042c0 con 0x7ff058107d70 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 msgr2=0x7ff05807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 --2- 172.21.15.49:0/1431418878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff054011850 tx=0x7ff054011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 shutdown_connections 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 --2- 172.21.15.49:0/1431418878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff058107d70 0x7ff05807db00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:20.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 >> 172.21.15.49:0/1431418878 conn(0x7ff058103350 msgr2=0x7ff058103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:20.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 shutdown_connections 2024-03-16T08:07:20.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:19.677+0000 7ff05f5ca700 1 -- 172.21.15.49:0/1431418878 wait complete. 2024-03-16T08:07:20.289 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-03-16T08:07:20.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:20.288+0000 7f670fb62200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:07:20.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:20.481+0000 7f670fb62200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:07:21.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:20.778+0000 7f670fb62200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:07:21.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:21.063+0000 7f670fb62200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:07:21.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:21.309+0000 7f670fb62200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:07:22.403 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:22.145+0000 7f670fb62200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:07:23.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:22 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1395979110' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:07:23.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:22.743+0000 7f670fb62200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:07:23.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:22.847+0000 7f670fb62200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:07:23.342 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:23.073+0000 7f670fb62200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:07:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:07:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:07:23.356 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:07:23.357 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:07:09.045494+0000", 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1b0bc7700 1 Processor -- start 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1b0bc7700 1 -- start start 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1b0bc7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:23.358 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1b0bc7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac07bdc0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47780/0 (socket says 172.21.15.49:47780) 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.689+0000 7fb1aa59c700 1 -- 172.21.15.49:0/3900932577 learned_addr learned my addr 172.21.15.49:0/3900932577 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.690+0000 7fb1aa59c700 1 -- 172.21.15.49:0/3900932577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac106f00 con 0x7fb1ac1065a0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.690+0000 7fb1aa59c700 1 --2- 172.21.15.49:0/3900932577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098a0 tx=0x7fb19c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84633e288d81048c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.690+0000 7fb1a959a700 1 -- 172.21.15.49:0/3900932577 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb19c004160 con 0x7fb1ac1065a0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.690+0000 7fb1a959a700 1 -- 172.21.15.49:0/3900932577 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb19c004300 con 0x7fb1ac1065a0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.690+0000 7fb1a959a700 1 -- 172.21.15.49:0/3900932577 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb19c011b50 con 0x7fb1ac1065a0 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/3900932577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 msgr2=0x7fb1ac1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 --2- 172.21.15.49:0/3900932577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098a0 tx=0x7fb19c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/3900932577 shutdown_connections 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 --2- 172.21.15.49:0/3900932577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac1069c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/3900932577 >> 172.21.15.49:0/3900932577 conn(0x7fb1ac102160 msgr2=0x7fb1ac1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:23.359 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/3900932577 shutdown_connections 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.691+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/3900932577 wait complete. 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1b0bc7700 1 Processor -- start 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1b0bc7700 1 -- start start 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1b0bc7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1b0bc7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac107f20 con 0x7fb1ac1065a0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47790/0 (socket says 172.21.15.49:47790) 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1aa59c700 1 -- 172.21.15.49:0/1395979110 learned_addr learned my addr 172.21.15.49:0/1395979110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.692+0000 7fb1aa59c700 1 -- 172.21.15.49:0/1395979110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac19e340 con 0x7fb1ac1065a0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1aa59c700 1 --2- 172.21.15.49:0/1395979110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb19c011b60 tx=0x7fb19c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb19c003b20 con 0x7fb1ac1065a0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb19c003cc0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac19e5d0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.360 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb19c024840 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac1a12c0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb19c022020 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.693+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb19c011e00 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.694+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac0693a0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.696+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb19c02d3f0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.769+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb1ac0008d0 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.770+0000 7fb1a37fe700 1 -- 172.21.15.49:0/1395979110 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb19c032030 con 0x7fb1ac1065a0 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 msgr2=0x7fb1ac19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 --2- 172.21.15.49:0/1395979110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb19c011b60 tx=0x7fb19c011c40 comp rx=0 tx=0).stop 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 shutdown_connections 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 --2- 172.21.15.49:0/1395979110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1ac1065a0 0x7fb1ac19de00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 >> 172.21.15.49:0/1395979110 conn(0x7fb1ac102160 msgr2=0x7fb1ac1949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 shutdown_connections 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:22.771+0000 7fb1b0bc7700 1 -- 172.21.15.49:0/1395979110 wait complete. 2024-03-16T08:07:23.361 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (3/15)... 2024-03-16T08:07:24.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: Activating manager daemon x 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: mgrmap e2: x(active, starting, since 0.00323001s) 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: Manager daemon x is now available 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' 2024-03-16T08:07:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:23 smithi049 ceph-mon[31942]: from='mgr.14100 172.21.15.49:0/468931862' entity='mgr.x' 2024-03-16T08:07:25.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:25 smithi049 ceph-mon[31942]: mgrmap e3: x(active, since 1.00715s) 2024-03-16T08:07:26.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:26 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2295680359' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:07:26.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:07:26.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:07:26.781 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:07:26.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:26.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:07:09.045494+0000", 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cf940700 1 Processor -- start 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cf940700 1 -- start start 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cf940700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cf940700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81075d0 con 0x7fc5c8106be0 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cd6dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cd6dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47880/0 (socket says 172.21.15.49:47880) 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.765+0000 7fc5cd6dc700 1 -- 172.21.15.49:0/2849404152 learned_addr learned my addr 172.21.15.49:0/2849404152 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.766+0000 7fc5cd6dc700 1 -- 172.21.15.49:0/2849404152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c8107de0 con 0x7fc5c8106be0 2024-03-16T08:07:26.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.766+0000 7fc5cd6dc700 1 --2- 172.21.15.49:0/2849404152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40098a0 tx=0x7fc5c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef292ae332f2a58 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.766+0000 7fc5bffff700 1 -- 172.21.15.49:0/2849404152 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c4004160 con 0x7fc5c8106be0 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.766+0000 7fc5bffff700 1 -- 172.21.15.49:0/2849404152 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5c4004300 con 0x7fc5c8106be0 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.766+0000 7fc5bffff700 1 -- 172.21.15.49:0/2849404152 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c4011b50 con 0x7fc5c8106be0 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 -- 172.21.15.49:0/2849404152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 msgr2=0x7fc5c8107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2849404152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40098a0 tx=0x7fc5c4009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 -- 172.21.15.49:0/2849404152 shutdown_connections 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2849404152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c8107000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 -- 172.21.15.49:0/2849404152 >> 172.21.15.49:0/2849404152 conn(0x7fc5c81021a0 msgr2=0x7fc5c81045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 -- 172.21.15.49:0/2849404152 shutdown_connections 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.767+0000 7fc5cf940700 1 -- 172.21.15.49:0/2849404152 wait complete. 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cf940700 1 Processor -- start 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cf940700 1 -- start start 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cf940700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cf940700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c807af60 con 0x7fc5c8106be0 2024-03-16T08:07:26.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cd6dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cd6dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47882/0 (socket says 172.21.15.49:47882) 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cd6dc700 1 -- 172.21.15.49:0/2295680359 learned_addr learned my addr 172.21.15.49:0/2295680359 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.768+0000 7fc5cd6dc700 1 -- 172.21.15.49:0/2295680359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c810cfe0 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5cd6dc700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009fb0 tx=0x7fc5c4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c40038b0 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5c4011e50 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c810b1d0 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c4019550 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.769+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c810b710 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 3) ==== 45989+0+0 (secure 0 0 0) 0x7fc5c4019800 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5be7fc700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 0x7fc5b403b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc5c404cff0 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c80693a0 con 0x7fc5c8106be0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5ccedb700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 0x7fc5b403b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:26.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.770+0000 7fc5ccedb700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 0x7fc5b403b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:25.775+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc5c4003ac0 con 0x7fc5c8106be0 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.066+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5c8195520 con 0x7fc5c8106be0 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.067+0000 7fc5be7fc700 1 -- 172.21.15.49:0/2295680359 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc5c4032030 con 0x7fc5c8106be0 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 msgr2=0x7fc5b403b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 0x7fc5b403b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 msgr2=0x7fc5c810caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009fb0 tx=0x7fc5c4011bf0 comp rx=0 tx=0).stop 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 shutdown_connections 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fc5b4038dc0 0x7fc5b403b280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 --2- 172.21.15.49:0/2295680359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c8106be0 0x7fc5c810caa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 >> 172.21.15.49:0/2295680359 conn(0x7fc5c81021a0 msgr2=0x7fc5c8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 shutdown_connections 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:26.069+0000 7fc5cf940700 1 -- 172.21.15.49:0/2295680359 wait complete. 2024-03-16T08:07:26.794 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-03-16T08:07:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:27 smithi049 ceph-mon[31942]: mgrmap e4: x(active, since 3s) 2024-03-16T08:07:27.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:27.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T08:07:27.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:27.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T08:07:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.169+0000 7ff3646b2700 1 Processor -- start 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.169+0000 7ff3646b2700 1 -- start start 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff3646b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff3646b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c109ca0 con 0x7ff35c1092b0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36244e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36244e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47884/0 (socket says 172.21.15.49:47884) 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36244e700 1 -- 172.21.15.49:0/3697654687 learned_addr learned my addr 172.21.15.49:0/3697654687 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36244e700 1 -- 172.21.15.49:0/3697654687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c109e20 con 0x7ff35c1092b0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36244e700 1 --2- 172.21.15.49:0/3697654687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3580098a0 tx=0x7ff358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6cac3f478f9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36144c700 1 -- 172.21.15.49:0/3697654687 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff358004160 con 0x7ff35c1092b0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36144c700 1 -- 172.21.15.49:0/3697654687 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff358004300 con 0x7ff35c1092b0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.170+0000 7ff36144c700 1 -- 172.21.15.49:0/3697654687 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff358011b50 con 0x7ff35c1092b0 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 -- 172.21.15.49:0/3697654687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 msgr2=0x7ff35c1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3697654687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3580098a0 tx=0x7ff358009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 -- 172.21.15.49:0/3697654687 shutdown_connections 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3697654687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c1096d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 -- 172.21.15.49:0/3697654687 >> 172.21.15.49:0/3697654687 conn(0x7ff35c101910 msgr2=0x7ff35c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 -- 172.21.15.49:0/3697654687 shutdown_connections 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.171+0000 7ff3646b2700 1 -- 172.21.15.49:0/3697654687 wait complete. 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff3646b2700 1 Processor -- start 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff3646b2700 1 -- start start 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff3646b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff3646b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c106aa0 con 0x7ff35c1092b0 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff36244e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff36244e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47888/0 (socket says 172.21.15.49:47888) 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff36244e700 1 -- 172.21.15.49:0/3967202217 learned_addr learned my addr 172.21.15.49:0/3967202217 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff36244e700 1 -- 172.21.15.49:0/3967202217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c19c120 con 0x7ff35c1092b0 2024-03-16T08:07:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.172+0000 7ff36244e700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff358000c00 tx=0x7ff358011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff358003b20 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff358003cc0 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c19c410 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff358024910 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c19f0a0 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.173+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7ff358022020 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.174+0000 7ff3537fe700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 0x7ff34803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.174+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c0693a0 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.174+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff358011e00 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.174+0000 7ff361c4d700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 0x7ff34803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.174+0000 7ff361c4d700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 0x7ff34803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006fd0 tx=0x7ff34c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.179+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff35800b3e0 con 0x7ff35c1092b0 2024-03-16T08:07:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.381+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff35c19f270 con 0x7ff35c1092b0 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.382+0000 7ff3537fe700 1 -- 172.21.15.49:0/3967202217 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7ff35801a960 con 0x7ff35c1092b0 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.385+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 msgr2=0x7ff34803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.385+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 0x7ff34803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006fd0 tx=0x7ff34c006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.385+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 msgr2=0x7ff35c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.385+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff358000c00 tx=0x7ff358011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 shutdown_connections 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7ff348038f20 0x7ff34803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 --2- 172.21.15.49:0/3967202217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff35c1092b0 0x7ff35c19bbe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 >> 172.21.15.49:0/3967202217 conn(0x7ff35c101910 msgr2=0x7ff35c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 shutdown_connections 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:27.386+0000 7ff3646b2700 1 -- 172.21.15.49:0/3967202217 wait complete. 2024-03-16T08:07:27.952 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-03-16T08:07:28.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:28 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3967202217' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T08:07:29.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:29 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/702985997' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-16T08:07:29.403 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setuser ceph since I am not root 2024-03-16T08:07:29.403 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setgroup ceph since I am not root 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.312+0000 7f1cd3e82700 1 Processor -- start 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.312+0000 7f1cd3e82700 1 -- start start 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.312+0000 7f1cd3e82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.312+0000 7f1cd3e82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc108bb0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd1c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd1c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47896/0 (socket says 172.21.15.49:47896) 2024-03-16T08:07:29.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd1c1e700 1 -- 172.21.15.49:0/1382679468 learned_addr learned my addr 172.21.15.49:0/1382679468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd1c1e700 1 -- 172.21.15.49:0/1382679468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc106e20 con 0x7f1ccc1081c0 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd1c1e700 1 --2- 172.21.15.49:0/1382679468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009e80 tx=0x7f1cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9507c2918cd116f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd0c1c700 1 -- 172.21.15.49:0/1382679468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4017070 con 0x7f1ccc1081c0 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd0c1c700 1 -- 172.21.15.49:0/1382679468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cc400b8c0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.313+0000 7f1cd0c1c700 1 -- 172.21.15.49:0/1382679468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc40045c0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 -- 172.21.15.49:0/1382679468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 msgr2=0x7f1ccc1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/1382679468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009e80 tx=0x7f1cc400b330 comp rx=0 tx=0).stop 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 -- 172.21.15.49:0/1382679468 shutdown_connections 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/1382679468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc1085e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 -- 172.21.15.49:0/1382679468 >> 172.21.15.49:0/1382679468 conn(0x7f1ccc101930 msgr2=0x7f1ccc103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 -- 172.21.15.49:0/1382679468 shutdown_connections 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.314+0000 7f1cd3e82700 1 -- 172.21.15.49:0/1382679468 wait complete. 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd3e82700 1 Processor -- start 2024-03-16T08:07:29.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd3e82700 1 -- start start 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd3e82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd3e82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc10b1e0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd1c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd1c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47902/0 (socket says 172.21.15.49:47902) 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd1c1e700 1 -- 172.21.15.49:0/702985997 learned_addr learned my addr 172.21.15.49:0/702985997 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd1c1e700 1 -- 172.21.15.49:0/702985997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc1922e0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.315+0000 7f1cd1c1e700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009530 tx=0x7f1cc400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4020020 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cc4003b50 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1904d0 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4017040 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc190a10 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f1cc4023020 con 0x7f1ccc1081c0 2024-03-16T08:07:29.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.316+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc063900 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.317+0000 7f1ccaffd700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 0x7f1cb403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.317+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1cc400f050 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.317+0000 7f1cd141d700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 0x7f1cb403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.317+0000 7f1cd141d700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 0x7f1cb403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.322+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1cc402a020 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:28.584+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1ccc195b40 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.098+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1cc401b370 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.099+0000 7f1ccaffd700 1 -- 172.21.15.49:0/702985997 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f1cc4025400 con 0x7f1ccc1081c0 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 msgr2=0x7f1cb403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 0x7f1cb403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 msgr2=0x7f1ccc191da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009530 tx=0x7f1cc400bd50 comp rx=0 tx=0).stop 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 shutdown_connections 2024-03-16T08:07:29.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f1cb4038f20 0x7f1cb403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:29.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 --2- 172.21.15.49:0/702985997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ccc1081c0 0x7f1ccc191da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:29.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 >> 172.21.15.49:0/702985997 conn(0x7f1ccc101930 msgr2=0x7f1ccc102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:29.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 shutdown_connections 2024-03-16T08:07:29.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:29.101+0000 7f1cd3e82700 1 -- 172.21.15.49:0/702985997 wait complete. 2024-03-16T08:07:29.990 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:29.697+0000 7f0116373200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:07:29.990 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:29.798+0000 7f0116373200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:07:30.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/702985997' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-16T08:07:30.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-mon[31942]: mgrmap e5: x(active, since 6s) 2024-03-16T08:07:30.624 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:30.524+0000 7f0116373200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:07:30.624 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:30.624+0000 7f0116373200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:07:30.874 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:30.774+0000 7f0116373200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:07:30.874 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:30.875+0000 7f0116373200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:07:30.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.105+0000 7f3bb0e4e700 1 Processor -- start 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.105+0000 7f3bb0e4e700 1 -- start start 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3bb0e4e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3bb0e4e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac07be40 con 0x7f3bac108730 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47928/0 (socket says 172.21.15.49:47928) 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3baa59c700 1 -- 172.21.15.49:0/3519959734 learned_addr learned my addr 172.21.15.49:0/3519959734 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:30.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.106+0000 7f3baa59c700 1 -- 172.21.15.49:0/3519959734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac10b060 con 0x7f3bac108730 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.107+0000 7f3baa59c700 1 --2- 172.21.15.49:0/3519959734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098a0 tx=0x7f3b9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ae18f04f1962af server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.107+0000 7f3ba959a700 1 -- 172.21.15.49:0/3519959734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c004160 con 0x7f3bac108730 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.107+0000 7f3ba959a700 1 -- 172.21.15.49:0/3519959734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b9c004300 con 0x7f3bac108730 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.107+0000 7f3ba959a700 1 -- 172.21.15.49:0/3519959734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c011b50 con 0x7f3bac108730 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3519959734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 msgr2=0x7f3bac10ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3519959734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098a0 tx=0x7f3b9c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3519959734 shutdown_connections 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3519959734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac10ab20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3519959734 >> 172.21.15.49:0/3519959734 conn(0x7f3bac102160 msgr2=0x7f3bac1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3519959734 shutdown_connections 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.108+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3519959734 wait complete. 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3bb0e4e700 1 Processor -- start 2024-03-16T08:07:30.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3bb0e4e700 1 -- start start 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3bb0e4e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3bb0e4e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10c020 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3baa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47934/0 (socket says 172.21.15.49:47934) 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.109+0000 7f3baa59c700 1 -- 172.21.15.49:0/3411349982 learned_addr learned my addr 172.21.15.49:0/3411349982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3baa59c700 1 -- 172.21.15.49:0/3411349982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac19c140 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3baa59c700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011b60 tx=0x7f3b9c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c003b20 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b9c003cc0 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac19c3d0 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.110+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac19f0c0 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c024840 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f3b9c022020 con 0x7f3bac108730 2024-03-16T08:07:30.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3b9b7fe700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 0x7f3b9403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b9c011e00 con 0x7f3bac108730 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3ba9d9b700 1 -- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 msgr2=0x7f3b9403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.111+0000 7f3ba9d9b700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 0x7f3b9403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.112+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac19c8b0 con 0x7f3bac108730 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.121+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3bac19c8b0 con 0x7f3bac108730 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.312+0000 7f3ba9d9b700 1 -- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 msgr2=0x7f3b9403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.312+0000 7f3ba9d9b700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 0x7f3b9403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.386+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3bac19f5f0 con 0x7f3bac108730 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.386+0000 7f3b9b7fe700 1 -- 172.21.15.49:0/3411349982 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3b9c019c70 con 0x7f3bac108730 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 msgr2=0x7f3b9403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 0x7f3b9403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 msgr2=0x7f3bac19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:30.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011b60 tx=0x7f3b9c011c40 comp rx=0 tx=0).stop 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 shutdown_connections 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7f3b94038f20 0x7f3b9403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 --2- 172.21.15.49:0/3411349982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac108730 0x7f3bac19bc00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.389+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 >> 172.21.15.49:0/3411349982 conn(0x7f3bac102160 msgr2=0x7f3bac102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.390+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 shutdown_connections 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:30.390+0000 7f3bb0e4e700 1 -- 172.21.15.49:0/3411349982 wait complete. 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-16T08:07:30.998 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-03-16T08:07:31.221 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3411349982' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T08:07:31.221 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:30.963+0000 7f0116373200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:07:31.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:31.246+0000 7f0116373200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:07:31.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:31.362+0000 7f0116373200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:07:31.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:31.455+0000 7f0116373200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:07:32.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:31.555+0000 7f0116373200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:07:32.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:31.736+0000 7f0116373200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:07:33.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:32 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:32.751+0000 7f0116373200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:07:33.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:32 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:32.887+0000 7f0116373200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:07:33.529 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:33 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:33.233+0000 7f0116373200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:07:33.529 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:33 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:33.326+0000 7f0116373200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:07:33.529 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:33 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:33.530+0000 7f0116373200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:07:33.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:33 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:33.723+0000 7f0116373200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:07:34.285 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:34 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:34.018+0000 7f0116373200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:07:34.285 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:34 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:34.286+0000 7f0116373200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:07:34.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:34 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:34.532+0000 7f0116373200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:07:35.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:35 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:35.370+0000 7f0116373200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:07:36.286 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:35 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:35.963+0000 7f0116373200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:07:36.286 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:36.065+0000 7f0116373200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:07:36.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:36.292+0000 7f0116373200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:07:36.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: Active manager daemon x restarted 2024-03-16T08:07:36.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: Activating manager daemon x 2024-03-16T08:07:36.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: osdmap e2: 0 total, 0 up, 0 in 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: mgrmap e6: x(active, starting, since 0.00341913s) 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: Manager daemon x is now available 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:36 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: Found migration_current of "None". Setting to last migration. 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:37.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:37 smithi049 ceph-mon[31942]: mgrmap e7: x(active, since 1.00537s) 2024-03-16T08:07:37.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:37.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda95e8e700 1 Processor -- start 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda95e8e700 1 -- start start 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda95e8e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda95e8e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda9007bde0 con 0x7fda90108820 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda8f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda8f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47946/0 (socket says 172.21.15.49:47946) 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda8f7fe700 1 -- 172.21.15.49:0/764620544 learned_addr learned my addr 172.21.15.49:0/764620544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.451+0000 7fda8f7fe700 1 -- 172.21.15.49:0/764620544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9010b150 con 0x7fda90108820 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda8f7fe700 1 --2- 172.21.15.49:0/764620544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fda800098a0 tx=0x7fda80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72ab07daaad8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda8e7fc700 1 -- 172.21.15.49:0/764620544 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda80004160 con 0x7fda90108820 2024-03-16T08:07:37.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda8e7fc700 1 -- 172.21.15.49:0/764620544 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda80004300 con 0x7fda90108820 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda8e7fc700 1 -- 172.21.15.49:0/764620544 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda80011b50 con 0x7fda90108820 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda95e8e700 1 -- 172.21.15.49:0/764620544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 msgr2=0x7fda9010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.452+0000 7fda95e8e700 1 --2- 172.21.15.49:0/764620544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fda800098a0 tx=0x7fda80009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 -- 172.21.15.49:0/764620544 shutdown_connections 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 --2- 172.21.15.49:0/764620544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda9010ac10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 -- 172.21.15.49:0/764620544 >> 172.21.15.49:0/764620544 conn(0x7fda90102230 msgr2=0x7fda90104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 -- 172.21.15.49:0/764620544 shutdown_connections 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 -- 172.21.15.49:0/764620544 wait complete. 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.453+0000 7fda95e8e700 1 Processor -- start 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda95e8e700 1 -- start start 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda95e8e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda95e8e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda9010c040 con 0x7fda90108820 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda8f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda8f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47950/0 (socket says 172.21.15.49:47950) 2024-03-16T08:07:37.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda8f7fe700 1 -- 172.21.15.49:0/1563443201 learned_addr learned my addr 172.21.15.49:0/1563443201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda8f7fe700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda901085e0 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.454+0000 7fda8f7fe700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda80011670 tx=0x7fda80011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.455+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda800039c0 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.455+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda901067d0 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.455+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda80011e50 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.455+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda8001a5c0 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.455+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda90106d10 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fda8001a8c0 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8cff9700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda7803baa0 con 0x7fda78038ed0 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda8004d120 con 0x7fda90108820 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8effd700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.456+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:07:37.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.657+0000 7fda8effd700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:31.657+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:32.058+0000 7fda8effd700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:32.058+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:32.859+0000 7fda8effd700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:32.859+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:34.460+0000 7fda8effd700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1374126004 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:34.460+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:36.307+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7fda8001abb0 con 0x7fda90108820 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:36.307+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 msgr2=0x7fda7803b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:36.307+0000 7fda8cff9700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.308+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7fda8004edc0 con 0x7fda90108820 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.308+0000 7fda8cff9700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 0x7fda7803ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.308+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda80008ea0 con 0x7fda7803c7c0 2024-03-16T08:07:37.854 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.308+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 0x7fda7803ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.309+0000 7fda8effd700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 0x7fda7803ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda84003a10 tx=0x7fda84009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.310+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fda80008ea0 con 0x7fda7803c7c0 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.316+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fda900693a0 con 0x7fda7803c7c0 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda8cff9700 1 -- 172.21.15.49:0/1563443201 <== mgr.14120 v2:172.21.15.49:6800/2659026035 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fda900693a0 con 0x7fda7803c7c0 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 msgr2=0x7fda7803ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 0x7fda7803ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda84003a10 tx=0x7fda84009290 comp rx=0 tx=0).stop 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 msgr2=0x7fda901080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda80011670 tx=0x7fda80011b60 comp rx=0 tx=0).stop 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 shutdown_connections 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fda7803c7c0 0x7fda7803ebb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda90108820 0x7fda901080a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 --2- 172.21.15.49:0/1563443201 >> [v2:172.21.15.49:6800/1374126004,v1:172.21.15.49:6801/1374126004] conn(0x7fda78038ed0 0x7fda7803b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 >> 172.21.15.49:0/1563443201 conn(0x7fda90102230 msgr2=0x7fda90102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:37.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 shutdown_connections 2024-03-16T08:07:37.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:37.317+0000 7fda95e8e700 1 -- 172.21.15.49:0/1563443201 wait complete. 2024-03-16T08:07:37.856 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-03-16T08:07:37.856 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-03-16T08:07:38.517 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:38 smithi049 ceph-mon[31942]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T08:07:38.517 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:38 smithi049 ceph-mon[31942]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T08:07:38.517 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:38 smithi049 ceph-mon[31942]: [16/Mar/2024:08:07:38] ENGINE Bus STARTING 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd089ed9700 1 Processor -- start 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd089ed9700 1 -- start start 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd089ed9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd089ed9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841086e0 con 0x7fd084107cf0 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59922/0 (socket says 172.21.15.49:59922) 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd0837fe700 1 -- 172.21.15.49:0/3605766613 learned_addr learned my addr 172.21.15.49:0/3605766613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:39.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.281+0000 7fd0837fe700 1 -- 172.21.15.49:0/3605766613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd084108ea0 con 0x7fd084107cf0 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd0837fe700 1 --2- 172.21.15.49:0/3605766613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0740098a0 tx=0x7fd074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e099a6960caca1e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd0827fc700 1 -- 172.21.15.49:0/3605766613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd074004160 con 0x7fd084107cf0 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd0827fc700 1 -- 172.21.15.49:0/3605766613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd074004300 con 0x7fd084107cf0 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd0827fc700 1 -- 172.21.15.49:0/3605766613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd074011b50 con 0x7fd084107cf0 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 -- 172.21.15.49:0/3605766613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 msgr2=0x7fd084108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 --2- 172.21.15.49:0/3605766613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0740098a0 tx=0x7fd074009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 -- 172.21.15.49:0/3605766613 shutdown_connections 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 --2- 172.21.15.49:0/3605766613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd084108110 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 -- 172.21.15.49:0/3605766613 >> 172.21.15.49:0/3605766613 conn(0x7fd084103270 msgr2=0x7fd0841056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 -- 172.21.15.49:0/3605766613 shutdown_connections 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.282+0000 7fd089ed9700 1 -- 172.21.15.49:0/3605766613 wait complete. 2024-03-16T08:07:39.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.283+0000 7fd089ed9700 1 Processor -- start 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.283+0000 7fd089ed9700 1 -- start start 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.283+0000 7fd089ed9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.283+0000 7fd089ed9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd08410c250 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59930/0 (socket says 172.21.15.49:59930) 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd0837fe700 1 -- 172.21.15.49:0/2237417811 learned_addr learned my addr 172.21.15.49:0/2237417811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd0837fe700 1 -- 172.21.15.49:0/2237417811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08407e040 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd0837fe700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd074011850 tx=0x7fd074011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd074003af0 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd074020070 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0740195a0 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08407c230 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.284+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08407c770 con 0x7fd084107cf0 2024-03-16T08:07:39.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.285+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7fd07401a430 con 0x7fd084107cf0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.285+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084063900 con 0x7fd084107cf0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.285+0000 7fd080ff9700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 0x7fd06c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.285+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd074011e00 con 0x7fd084107cf0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.285+0000 7fd082ffd700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 0x7fd06c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.286+0000 7fd082ffd700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 0x7fd06c03b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.290+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd074019800 con 0x7fd084107cf0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.359+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fd07402bd00 con 0x7fd084107cf0 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.517+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd084104500 con 0x7fd06c038e50 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.523+0000 7fd080ff9700 1 -- 172.21.15.49:0/2237417811 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd084104500 con 0x7fd06c038e50 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 msgr2=0x7fd06c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 0x7fd06c03b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 msgr2=0x7fd08407db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:39.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd074011850 tx=0x7fd074011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 shutdown_connections 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd06c038e50 0x7fd06c03b310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 --2- 172.21.15.49:0/2237417811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd084107cf0 0x7fd08407db00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 >> 172.21.15.49:0/2237417811 conn(0x7fd084103270 msgr2=0x7fd084103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 shutdown_connections 2024-03-16T08:07:39.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:38.525+0000 7fd089ed9700 1 -- 172.21.15.49:0/2237417811 wait complete. 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: [16/Mar/2024:08:07:38] ENGINE Serving on http://172.21.15.49:8765 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: [16/Mar/2024:08:07:38] ENGINE Serving on https://172.21.15.49:7150 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: [16/Mar/2024:08:07:38] ENGINE Bus STARTED 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: mgrmap e8: x(active, since 2s) 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:39.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:39.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:39 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:40.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:40 smithi049 ceph-mon[31942]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.607+0000 7f3aa0cc6700 1 Processor -- start 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.607+0000 7f3aa0cc6700 1 -- start start 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.607+0000 7f3aa0cc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.607+0000 7f3aa0cc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c07c7b0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59938/0 (socket says 172.21.15.49:59938) 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9a59c700 1 -- 172.21.15.49:0/3958512111 learned_addr learned my addr 172.21.15.49:0/3958512111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9a59c700 1 -- 172.21.15.49:0/3958512111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c07c930 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9a59c700 1 --2- 172.21.15.49:0/3958512111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0098a0 tx=0x7f3a8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec30c7b9bc516c87 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9959a700 1 -- 172.21.15.49:0/3958512111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c004160 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9959a700 1 -- 172.21.15.49:0/3958512111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c004300 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.608+0000 7f3a9959a700 1 -- 172.21.15.49:0/3958512111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c011b50 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/3958512111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 msgr2=0x7f3a9c07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/3958512111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0098a0 tx=0x7f3a8c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/3958512111 shutdown_connections 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/3958512111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c07c270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/3958512111 >> 172.21.15.49:0/3958512111 conn(0x7f3a9c1032b0 msgr2=0x7f3a9c1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/3958512111 shutdown_connections 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.609+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/3958512111 wait complete. 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3aa0cc6700 1 Processor -- start 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3aa0cc6700 1 -- start start 2024-03-16T08:07:40.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3aa0cc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3aa0cc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c10e1e0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3a9a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3a9a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59954/0 (socket says 172.21.15.49:59954) 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.610+0000 7f3a9a59c700 1 -- 172.21.15.49:0/1623194586 learned_addr learned my addr 172.21.15.49:0/1623194586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3a9a59c700 1 -- 172.21.15.49:0/1623194586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c19e2e0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3a9a59c700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c011b60 tx=0x7f3a8c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c003b20 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c003cc0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c19e570 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c024840 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.611+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c1a1260 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.612+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f3a8c022020 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.612+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9c0693a0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.613+0000 7f3a8b7fe700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 0x7f3a8403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.613+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a8c011e00 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.613+0000 7f3a99d9b700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 0x7f3a8403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.613+0000 7f3a99d9b700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 0x7f3a8403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006fd0 tx=0x7f3a90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.618+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a8c00b9e0 con 0x7f3a9c07ddc0 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.824+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3a9c1044b0 con 0x7f3a84038f70 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.825+0000 7f3a8b7fe700 1 -- 172.21.15.49:0/1623194586 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3a9c1044b0 con 0x7f3a84038f70 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 msgr2=0x7f3a8403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 0x7f3a8403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006fd0 tx=0x7f3a90006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 msgr2=0x7f3a9c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c011b60 tx=0x7f3a8c011c40 comp rx=0 tx=0).stop 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 shutdown_connections 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f3a84038f70 0x7f3a8403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 --2- 172.21.15.49:0/1623194586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c07ddc0 0x7f3a9c19dda0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 >> 172.21.15.49:0/1623194586 conn(0x7f3a9c1032b0 msgr2=0x7f3a9c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:40.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 shutdown_connections 2024-03-16T08:07:40.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:39.827+0000 7f3aa0cc6700 1 -- 172.21.15.49:0/1623194586 wait complete. 2024-03-16T08:07:40.417 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: Generating public/private rsa key pair. 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: Your identification has been saved in /tmp/tmpoov1osv5/key. 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: Your public key has been saved in /tmp/tmpoov1osv5/key.pub. 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: The key fingerprint is: 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: SHA256:30fZttFogn0ZIt4IpZSL0lj7GtT1J+R3sk8Feb+9xZQ ceph-053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: The key's randomart image is: 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: +---[RSA 3072]----+ 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | ... . | 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | ...o. . o .| 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | + +oo.+. .oo| 2024-03-16T08:07:41.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | o = oo *+.+E*| 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | o .S + ++BBB| 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | . .. . =.oB| 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | o . . .+o| 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | . . ..| 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: | | 2024-03-16T08:07:41.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:41 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: +----[SHA256]-----+ 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.783+0000 7fdd88f8b700 1 Processor -- start 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.783+0000 7fdd88f8b700 1 -- start start 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.783+0000 7fdd88f8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.783+0000 7fdd88f8b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8407be40 con 0x7fdd84108730 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59966/0 (socket says 172.21.15.49:59966) 2024-03-16T08:07:41.714 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8259c700 1 -- 172.21.15.49:0/4113089073 learned_addr learned my addr 172.21.15.49:0/4113089073 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8259c700 1 -- 172.21.15.49:0/4113089073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8410b060 con 0x7fdd84108730 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8259c700 1 --2- 172.21.15.49:0/4113089073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009e80 tx=0x7fdd7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db6cdc70225a6d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8159a700 1 -- 172.21.15.49:0/4113089073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c017070 con 0x7fdd84108730 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8159a700 1 -- 172.21.15.49:0/4113089073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd7c00b8c0 con 0x7fdd84108730 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.784+0000 7fdd8159a700 1 -- 172.21.15.49:0/4113089073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c0045c0 con 0x7fdd84108730 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4113089073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 msgr2=0x7fdd8410ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4113089073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009e80 tx=0x7fdd7c00b330 comp rx=0 tx=0).stop 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4113089073 shutdown_connections 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4113089073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd8410ab20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4113089073 >> 172.21.15.49:0/4113089073 conn(0x7fdd84102160 msgr2=0x7fdd841045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4113089073 shutdown_connections 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.785+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4113089073 wait complete. 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd88f8b700 1 Processor -- start 2024-03-16T08:07:41.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd88f8b700 1 -- start start 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd88f8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd88f8b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8410c020 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59970/0 (socket says 172.21.15.49:59970) 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd8259c700 1 -- 172.21.15.49:0/4052206530 learned_addr learned my addr 172.21.15.49:0/4052206530 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd8259c700 1 -- 172.21.15.49:0/4052206530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd841084f0 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd8259c700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004350 tx=0x7fdd7c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.786+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c020020 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.787+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd7c003c10 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.787+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd841066e0 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.787+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c017040 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.787+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd84106c20 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.787+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fdd7c023020 con 0x7fdd84108730 2024-03-16T08:07:41.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.788+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd840693a0 con 0x7fdd84108730 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.788+0000 7fdd7b7fe700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 0x7fdd7003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.788+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdd7c00f050 con 0x7fdd84108730 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.788+0000 7fdd81d9b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 0x7fdd7003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.788+0000 7fdd81d9b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 0x7fdd7003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.793+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdd7c02a020 con 0x7fdd84108730 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:40.996+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fdd841033b0 con 0x7fdd70038f70 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.117+0000 7fdd7b7fe700 1 -- 172.21.15.49:0/4052206530 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd841033b0 con 0x7fdd70038f70 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 msgr2=0x7fdd7003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 0x7fdd7003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 msgr2=0x7fdd84107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004350 tx=0x7fdd7c00bd50 comp rx=0 tx=0).stop 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 shutdown_connections 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fdd70038f70 0x7fdd7003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 --2- 172.21.15.49:0/4052206530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84108730 0x7fdd84107fb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:41.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 >> 172.21.15.49:0/4052206530 conn(0x7fdd84102160 msgr2=0x7fdd84102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:41.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 shutdown_connections 2024-03-16T08:07:41.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:41.119+0000 7fdd88f8b700 1 -- 172.21.15.49:0/4052206530 wait complete. 2024-03-16T08:07:42.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:42 smithi049 ceph-mon[31942]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:42.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:42 smithi049 ceph-mon[31942]: Generating ssh key... 2024-03-16T08:07:42.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:42 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:42.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:42 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd79292b700 1 Processor -- start 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd79292b700 1 -- start start 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd79292b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd79292b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1075b0 con 0x7fd78c106bc0 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59984/0 (socket says 172.21.15.49:59984) 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd78bfff700 1 -- 172.21.15.49:0/232340639 learned_addr learned my addr 172.21.15.49:0/232340639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:42.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.112+0000 7fd78bfff700 1 -- 172.21.15.49:0/232340639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c107dc0 con 0x7fd78c106bc0 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.113+0000 7fd78bfff700 1 --2- 172.21.15.49:0/232340639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098a0 tx=0x7fd77c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fb628ca01bcb04 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.115+0000 7fd78affd700 1 -- 172.21.15.49:0/232340639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd77c004160 con 0x7fd78c106bc0 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.115+0000 7fd78affd700 1 -- 172.21.15.49:0/232340639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd77c004300 con 0x7fd78c106bc0 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.115+0000 7fd78affd700 1 -- 172.21.15.49:0/232340639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd77c011b50 con 0x7fd78c106bc0 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- 172.21.15.49:0/232340639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 msgr2=0x7fd78c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 --2- 172.21.15.49:0/232340639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098a0 tx=0x7fd77c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- 172.21.15.49:0/232340639 shutdown_connections 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 --2- 172.21.15.49:0/232340639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c106fe0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- 172.21.15.49:0/232340639 >> 172.21.15.49:0/232340639 conn(0x7fd78c102160 msgr2=0x7fd78c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- 172.21.15.49:0/232340639 shutdown_connections 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- 172.21.15.49:0/232340639 wait complete. 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 Processor -- start 2024-03-16T08:07:42.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.116+0000 7fd79292b700 1 -- start start 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd79292b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd79292b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10b170 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59988/0 (socket says 172.21.15.49:59988) 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd78bfff700 1 -- 172.21.15.49:0/1363629743 learned_addr learned my addr 172.21.15.49:0/1363629743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd78bfff700 1 -- 172.21.15.49:0/1363629743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c19c7c0 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd78bfff700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000c00 tx=0x7fd77c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd77c003b20 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd77c003cc0 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c1a0dd0 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.117+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd77c024910 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.118+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a1170 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.118+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fd77c022020 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.119+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78c0693a0 con 0x7fd78c106bc0 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.119+0000 7fd7897fa700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 0x7fd77403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:42.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.119+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd77c011e00 con 0x7fd78c106bc0 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.119+0000 7fd78b7fe700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 0x7fd77403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.119+0000 7fd78b7fe700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 0x7fd77403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd780006fd0 tx=0x7fd780006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.124+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd77c0172b0 con 0x7fd78c106bc0 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.331+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd78c103470 con 0x7fd774038f70 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.331+0000 7fd7897fa700 1 -- 172.21.15.49:0/1363629743 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd78c103470 con 0x7fd774038f70 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 msgr2=0x7fd77403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 0x7fd77403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd780006fd0 tx=0x7fd780006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 msgr2=0x7fd78c07b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000c00 tx=0x7fd77c011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 shutdown_connections 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd774038f70 0x7fd77403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 --2- 172.21.15.49:0/1363629743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd78c106bc0 0x7fd78c07b9e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 >> 172.21.15.49:0/1363629743 conn(0x7fd78c102160 msgr2=0x7fd78c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 shutdown_connections 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:42.334+0000 7fd79292b700 1 -- 172.21.15.49:0/1363629743 wait complete. 2024-03-16T08:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-16T08:07:42.889 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-03-16T08:07:42.889 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-03-16T08:07:43.535 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:43 smithi049 ceph-mon[31942]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:44.349 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:44 smithi049 ceph-mon[31942]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:45.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:45 smithi049 ceph-mon[31942]: Deploying cephadm binary to smithi049 2024-03-16T08:07:48.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.314+0000 7f60ce0c0700 1 Processor -- start 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.314+0000 7f60ce0c0700 1 -- start start 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.314+0000 7f60ce0c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.314+0000 7f60ce0c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60c807be00 con 0x7f60c81087b0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49718/0 (socket says 172.21.15.49:49718) 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c77fe700 1 -- 172.21.15.49:0/3471866615 learned_addr learned my addr 172.21.15.49:0/3471866615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c77fe700 1 -- 172.21.15.49:0/3471866615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c810b0e0 con 0x7f60c81087b0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c77fe700 1 --2- 172.21.15.49:0/3471866615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009e80 tx=0x7f60b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df0b605f4ba6228 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c67fc700 1 -- 172.21.15.49:0/3471866615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b8017070 con 0x7f60c81087b0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c67fc700 1 -- 172.21.15.49:0/3471866615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60b800b8c0 con 0x7f60c81087b0 2024-03-16T08:07:48.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.315+0000 7f60c67fc700 1 -- 172.21.15.49:0/3471866615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b80045c0 con 0x7f60c81087b0 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 -- 172.21.15.49:0/3471866615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 msgr2=0x7f60c810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/3471866615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009e80 tx=0x7f60b800b330 comp rx=0 tx=0).stop 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 -- 172.21.15.49:0/3471866615 shutdown_connections 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/3471866615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c810aba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 -- 172.21.15.49:0/3471866615 >> 172.21.15.49:0/3471866615 conn(0x7f60c8102220 msgr2=0x7f60c8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 -- 172.21.15.49:0/3471866615 shutdown_connections 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 -- 172.21.15.49:0/3471866615 wait complete. 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.316+0000 7f60ce0c0700 1 Processor -- start 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60ce0c0700 1 -- start start 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60ce0c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60ce0c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60c810c090 con 0x7f60c81087b0 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49724/0 (socket says 172.21.15.49:49724) 2024-03-16T08:07:48.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60c77fe700 1 -- 172.21.15.49:0/382231719 learned_addr learned my addr 172.21.15.49:0/382231719 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60c77fe700 1 -- 172.21.15.49:0/382231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8108570 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.317+0000 7f60c77fe700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60b80072a0 tx=0x7f60b800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.318+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b8020020 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.318+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60b8003b50 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.318+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c8106760 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.318+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b8017040 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.318+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c8106ca0 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f60b8023020 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60c4ff9700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 0x7f60b403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c8063900 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f60b800f050 con 0x7f60c81087b0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60c6ffd700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 0x7f60b403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.319+0000 7f60c6ffd700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 0x7f60b403b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:48.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.324+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f60b8051070 con 0x7f60c81087b0 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:43.534+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) -- 0x7f60c8107500 con 0x7f60b4038f20 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.039+0000 7f60c4ff9700 1 -- 172.21.15.49:0/382231719 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f60c8107500 con 0x7f60b4038f20 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 msgr2=0x7f60b403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 0x7f60b403b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 msgr2=0x7f60c8108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60b80072a0 tx=0x7f60b800bd50 comp rx=0 tx=0).stop 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 shutdown_connections 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f60b4038f20 0x7f60b403b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 --2- 172.21.15.49:0/382231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c81087b0 0x7f60c8108030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 >> 172.21.15.49:0/382231719 conn(0x7f60c8102220 msgr2=0x7f60c8102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 shutdown_connections 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:48.042+0000 7f60ce0c0700 1 -- 172.21.15.49:0/382231719 wait complete. 2024-03-16T08:07:48.622 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2024-03-16T08:07:49.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:49 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:49.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:49 smithi049 ceph-mon[31942]: Added host smithi049 2024-03-16T08:07:49.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:49 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b436df700 1 Processor -- start 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b436df700 1 -- start start 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b436df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b436df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c07c860 con 0x7f1b3c07dde0 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b4147b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b4147b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49730/0 (socket says 172.21.15.49:49730) 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.076+0000 7f1b4147b700 1 -- 172.21.15.49:0/29468198 learned_addr learned my addr 172.21.15.49:0/29468198 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b4147b700 1 -- 172.21.15.49:0/29468198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c07c9e0 con 0x7f1b3c07dde0 2024-03-16T08:07:50.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b4147b700 1 --2- 172.21.15.49:0/29468198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098a0 tx=0x7f1b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd95676a811ed810 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b33fff700 1 -- 172.21.15.49:0/29468198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38004160 con 0x7f1b3c07dde0 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b33fff700 1 -- 172.21.15.49:0/29468198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b38004300 con 0x7f1b3c07dde0 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b33fff700 1 -- 172.21.15.49:0/29468198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38011b50 con 0x7f1b3c07dde0 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b436df700 1 -- 172.21.15.49:0/29468198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 msgr2=0x7f1b3c07c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b436df700 1 --2- 172.21.15.49:0/29468198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098a0 tx=0x7f1b38009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b436df700 1 -- 172.21.15.49:0/29468198 shutdown_connections 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.077+0000 7f1b436df700 1 --2- 172.21.15.49:0/29468198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c07c290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.078+0000 7f1b436df700 1 -- 172.21.15.49:0/29468198 >> 172.21.15.49:0/29468198 conn(0x7f1b3c103270 msgr2=0x7f1b3c1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.078+0000 7f1b436df700 1 -- 172.21.15.49:0/29468198 shutdown_connections 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.078+0000 7f1b436df700 1 -- 172.21.15.49:0/29468198 wait complete. 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.078+0000 7f1b436df700 1 Processor -- start 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.078+0000 7f1b436df700 1 -- start start 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b436df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b436df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c10e240 con 0x7f1b3c07dde0 2024-03-16T08:07:50.172 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b4147b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b4147b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49740/0 (socket says 172.21.15.49:49740) 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b4147b700 1 -- 172.21.15.49:0/241167647 learned_addr learned my addr 172.21.15.49:0/241167647 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b4147b700 1 -- 172.21.15.49:0/241167647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c19e360 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b4147b700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009fb0 tx=0x7f1b38011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b380038b0 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b38011e50 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c19e5f0 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38019420 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.079+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c1a12e0 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.080+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1b3801a430 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.080+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c063900 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.080+0000 7f1b327fc700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 0x7f1b2803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.080+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b3804d090 con 0x7f1b3c07dde0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.080+0000 7f1b40c7a700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 0x7f1b2803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:50.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.081+0000 7f1b40c7a700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 0x7f1b2803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.085+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b38016260 con 0x7f1b3c07dde0 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.312+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1b3c1044e0 con 0x7f1b28038f20 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.316+0000 7f1b327fc700 1 -- 172.21.15.49:0/241167647 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1b3c1044e0 con 0x7f1b28038f20 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 msgr2=0x7f1b2803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 0x7f1b2803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 msgr2=0x7f1b3c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009fb0 tx=0x7f1b38011bf0 comp rx=0 tx=0).stop 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 shutdown_connections 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f1b28038f20 0x7f1b2803b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 --2- 172.21.15.49:0/241167647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b3c07dde0 0x7f1b3c19de20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 >> 172.21.15.49:0/241167647 conn(0x7f1b3c103270 msgr2=0x7f1b3c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:50.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.318+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 shutdown_connections 2024-03-16T08:07:50.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:49.319+0000 7f1b436df700 1 -- 172.21.15.49:0/241167647 wait complete. 2024-03-16T08:07:50.175 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2024-03-16T08:07:50.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:50 smithi049 ceph-mon[31942]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:50.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:50 smithi049 ceph-mon[31942]: Saving service mon spec with placement count:5 2024-03-16T08:07:50.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:50 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:51.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:51 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:51.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:51 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:51.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:51 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:51.930 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-16T08:07:51.930 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.140+0000 7fc6def9a700 1 Processor -- start 2024-03-16T08:07:51.930 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.140+0000 7fc6def9a700 1 -- start start 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6def9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6def9a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d807bde0 con 0x7fc6d81087d0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6dcd36700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6dcd36700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49756/0 (socket says 172.21.15.49:49756) 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6dcd36700 1 -- 172.21.15.49:0/1578326206 learned_addr learned my addr 172.21.15.49:0/1578326206 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6dcd36700 1 -- 172.21.15.49:0/1578326206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d810b100 con 0x7fc6d81087d0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.141+0000 7fc6dcd36700 1 --2- 172.21.15.49:0/1578326206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0098a0 tx=0x7fc6cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fdbaa5e5dc98e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6d77fe700 1 -- 172.21.15.49:0/1578326206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc004160 con 0x7fc6d81087d0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6d77fe700 1 -- 172.21.15.49:0/1578326206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc004300 con 0x7fc6d81087d0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6d77fe700 1 -- 172.21.15.49:0/1578326206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc011b20 con 0x7fc6d81087d0 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 -- 172.21.15.49:0/1578326206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 msgr2=0x7fc6d810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:51.931 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 --2- 172.21.15.49:0/1578326206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0098a0 tx=0x7fc6cc009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 -- 172.21.15.49:0/1578326206 shutdown_connections 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 --2- 172.21.15.49:0/1578326206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d810abc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 -- 172.21.15.49:0/1578326206 >> 172.21.15.49:0/1578326206 conn(0x7fc6d8102220 msgr2=0x7fc6d8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 -- 172.21.15.49:0/1578326206 shutdown_connections 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.142+0000 7fc6def9a700 1 -- 172.21.15.49:0/1578326206 wait complete. 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6def9a700 1 Processor -- start 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6def9a700 1 -- start start 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6def9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6def9a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d810c110 con 0x7fc6d81087d0 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6dcd36700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6dcd36700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49760/0 (socket says 172.21.15.49:49760) 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.143+0000 7fc6dcd36700 1 -- 172.21.15.49:0/4036973881 learned_addr learned my addr 172.21.15.49:0/4036973881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:51.932 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6dcd36700 1 -- 172.21.15.49:0/4036973881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d819c1c0 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6dcd36700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0117e0 tx=0x7fc6cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc003ba0 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc01a910 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d819c450 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc0248f0 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.144+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d819c8b0 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.145+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d8063900 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.145+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fc6cc022020 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.146+0000 7fc6d5ffb700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 0x7fc6c803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.146+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc6cc04d3a0 con 0x7fc6d81087d0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.146+0000 7fc6d7fff700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 0x7fc6c803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.146+0000 7fc6d7fff700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 0x7fc6c803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:51.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.151+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6cc01a430 con 0x7fc6d81087d0 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.373+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc6d8103380 con 0x7fc6c8038f70 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.377+0000 7fc6d5ffb700 1 -- 172.21.15.49:0/4036973881 <== mgr.14120 v2:172.21.15.49:6800/2659026035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc6d8103380 con 0x7fc6c8038f70 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 msgr2=0x7fc6c803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 0x7fc6c803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 msgr2=0x7fc6d819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0117e0 tx=0x7fc6cc011ba0 comp rx=0 tx=0).stop 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 shutdown_connections 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fc6c8038f70 0x7fc6c803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 --2- 172.21.15.49:0/4036973881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d81087d0 0x7fc6d819bc80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 >> 172.21.15.49:0/4036973881 conn(0x7fc6d8102220 msgr2=0x7fc6d8102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 shutdown_connections 2024-03-16T08:07:51.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:51.380+0000 7fc6def9a700 1 -- 172.21.15.49:0/4036973881 wait complete. 2024-03-16T08:07:52.826 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:52 smithi049 ceph-mon[31942]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:07:52.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:52 smithi049 ceph-mon[31942]: Saving service mgr spec with placement count:2 2024-03-16T08:07:52.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:52 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2743321569' entity='client.admin' 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.329+0000 7f39c62c5700 1 Processor -- start 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39c62c5700 1 -- start start 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39c62c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39c62c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39c007bde0 con 0x7f39c0108790 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49770/0 (socket says 172.21.15.49:49770) 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39bffff700 1 -- 172.21.15.49:0/1732120700 learned_addr learned my addr 172.21.15.49:0/1732120700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39bffff700 1 -- 172.21.15.49:0/1732120700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c010b0c0 con 0x7f39c0108790 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39bffff700 1 --2- 172.21.15.49:0/1732120700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098a0 tx=0x7f39b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4cadc92b1c2dae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.330+0000 7f39beffd700 1 -- 172.21.15.49:0/1732120700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39b0004160 con 0x7f39c0108790 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39beffd700 1 -- 172.21.15.49:0/1732120700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b0004300 con 0x7f39c0108790 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39beffd700 1 -- 172.21.15.49:0/1732120700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39b0011b20 con 0x7f39c0108790 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 -- 172.21.15.49:0/1732120700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 msgr2=0x7f39c010ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 --2- 172.21.15.49:0/1732120700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098a0 tx=0x7f39b0009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 -- 172.21.15.49:0/1732120700 shutdown_connections 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 --2- 172.21.15.49:0/1732120700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c010ab80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 -- 172.21.15.49:0/1732120700 >> 172.21.15.49:0/1732120700 conn(0x7f39c0102180 msgr2=0x7f39c01045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 -- 172.21.15.49:0/1732120700 shutdown_connections 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.331+0000 7f39c62c5700 1 -- 172.21.15.49:0/1732120700 wait complete. 2024-03-16T08:07:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39c62c5700 1 Processor -- start 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39c62c5700 1 -- start start 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39c62c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39c62c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39c010c0d0 con 0x7f39c0108790 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49786/0 (socket says 172.21.15.49:49786) 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.332+0000 7f39bffff700 1 -- 172.21.15.49:0/2743321569 learned_addr learned my addr 172.21.15.49:0/2743321569 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39bffff700 1 -- 172.21.15.49:0/2743321569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c0108550 con 0x7f39c0108790 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39bffff700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39b00117e0 tx=0x7f39b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39b0003bf0 con 0x7f39c0108790 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b001a910 con 0x7f39c0108790 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c0106740 con 0x7f39c0108790 2024-03-16T08:07:53.634 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39b0024970 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.333+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c0106c00 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.334+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f39b0022020 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.334+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c00693a0 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.334+0000 7f39bd7fa700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 0x7f39a803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.334+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f39b004d330 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.334+0000 7f39bf7fe700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 0x7f39a803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.335+0000 7f39bf7fe700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 0x7f39a803b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006fd0 tx=0x7f39b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.339+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f39b0011e20 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.556+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f39c01074a0 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.559+0000 7f39bd7fa700 1 -- 172.21.15.49:0/2743321569 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f39b000ba30 con 0x7f39c0108790 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 msgr2=0x7f39a803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 0x7f39a803b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006fd0 tx=0x7f39b4006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 msgr2=0x7f39c0108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:53.635 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39b00117e0 tx=0x7f39b0011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 shutdown_connections 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f39a8038ed0 0x7f39a803b390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 --2- 172.21.15.49:0/2743321569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c0108790 0x7f39c0108010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 >> 172.21.15.49:0/2743321569 conn(0x7f39c0102180 msgr2=0x7f39c0102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 shutdown_connections 2024-03-16T08:07:53.636 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:52.561+0000 7f39c62c5700 1 -- 172.21.15.49:0/2743321569 wait complete. 2024-03-16T08:07:54.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:54 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f501252f700 1 Processor -- start 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f501252f700 1 -- start start 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f501252f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f501252f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f500c07bdc0 con 0x7f500c1087f0 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f500bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f500bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46970/0 (socket says 172.21.15.49:46970) 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.058+0000 7f500bfff700 1 -- 172.21.15.49:0/4073341246 learned_addr learned my addr 172.21.15.49:0/4073341246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.059+0000 7f500bfff700 1 -- 172.21.15.49:0/4073341246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c10b120 con 0x7f500c1087f0 2024-03-16T08:07:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.059+0000 7f500bfff700 1 --2- 172.21.15.49:0/4073341246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098a0 tx=0x7f4ffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ccb80d235c7678 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.059+0000 7f500affd700 1 -- 172.21.15.49:0/4073341246 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc004160 con 0x7f500c1087f0 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.059+0000 7f500affd700 1 -- 172.21.15.49:0/4073341246 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc004300 con 0x7f500c1087f0 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.059+0000 7f500affd700 1 -- 172.21.15.49:0/4073341246 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc011b20 con 0x7f500c1087f0 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 -- 172.21.15.49:0/4073341246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 msgr2=0x7f500c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 --2- 172.21.15.49:0/4073341246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098a0 tx=0x7f4ffc009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 -- 172.21.15.49:0/4073341246 shutdown_connections 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 --2- 172.21.15.49:0/4073341246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c10abe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 -- 172.21.15.49:0/4073341246 >> 172.21.15.49:0/4073341246 conn(0x7f500c102220 msgr2=0x7f500c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 -- 172.21.15.49:0/4073341246 shutdown_connections 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 -- 172.21.15.49:0/4073341246 wait complete. 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.060+0000 7f501252f700 1 Processor -- start 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f501252f700 1 -- start start 2024-03-16T08:07:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f501252f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f501252f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f500c10c120 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f500bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f500bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46976/0 (socket says 172.21.15.49:46976) 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f500bfff700 1 -- 172.21.15.49:0/309640071 learned_addr learned my addr 172.21.15.49:0/309640071 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.061+0000 7f500bfff700 1 -- 172.21.15.49:0/309640071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c1085b0 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f500bfff700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0117e0 tx=0x7f4ffc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc003bf0 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc01a910 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c1067a0 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc024970 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.062+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c106c60 con 0x7f500c1087f0 2024-03-16T08:07:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.063+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f4ffc022020 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.063+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c063900 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.063+0000 7f50097fa700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 0x7f4ff803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.063+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ffc04d180 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.063+0000 7f500b7fe700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 0x7f4ff803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.064+0000 7f500b7fe700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 0x7f4ff803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4006fd0 tx=0x7f4ff4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.068+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4ffc011e20 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.292+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f500c106ee0 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.294+0000 7f50097fa700 1 -- 172.21.15.49:0/309640071 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4ffc00ba30 con 0x7f500c1087f0 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 msgr2=0x7f4ff803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 0x7f4ff803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4006fd0 tx=0x7f4ff4006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 msgr2=0x7f500c108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0117e0 tx=0x7f4ffc011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 shutdown_connections 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f4ff8038f20 0x7f4ff803b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 --2- 172.21.15.49:0/309640071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c1087f0 0x7f500c108070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.296+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 >> 172.21.15.49:0/309640071 conn(0x7f500c102220 msgr2=0x7f500c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.297+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 shutdown_connections 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:54.297+0000 7f501252f700 1 -- 172.21.15.49:0/309640071 wait complete. 2024-03-16T08:07:54.851 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-03-16T08:07:55.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:55 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/309640071' entity='client.admin' 2024-03-16T08:07:55.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:55 smithi049 ceph-mon[31942]: from='mgr.14120 172.21.15.49:0/823763577' entity='mgr.x' 2024-03-16T08:07:56.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:56 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2430380071' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-16T08:07:56.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setuser ceph since I am not root 2024-03-16T08:07:56.536 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setgroup ceph since I am not root 2024-03-16T08:07:56.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4ec39700 1 Processor -- start 2024-03-16T08:07:56.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4ec39700 1 -- start start 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4ec39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4ec39700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4807be00 con 0x7f8e48108780 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4c9d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4c9d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46984/0 (socket says 172.21.15.49:46984) 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.238+0000 7f8e4c9d5700 1 -- 172.21.15.49:0/3248819342 learned_addr learned my addr 172.21.15.49:0/3248819342 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.239+0000 7f8e4c9d5700 1 -- 172.21.15.49:0/3248819342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4810b0b0 con 0x7f8e48108780 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.239+0000 7f8e4c9d5700 1 --2- 172.21.15.49:0/3248819342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0098a0 tx=0x7f8e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee47ebf3aebc4eb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.239+0000 7f8e477fe700 1 -- 172.21.15.49:0/3248819342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e3c004160 con 0x7f8e48108780 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.239+0000 7f8e477fe700 1 -- 172.21.15.49:0/3248819342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3c004300 con 0x7f8e48108780 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.239+0000 7f8e477fe700 1 -- 172.21.15.49:0/3248819342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e3c011b20 con 0x7f8e48108780 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- 172.21.15.49:0/3248819342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 msgr2=0x7f8e4810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/3248819342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0098a0 tx=0x7f8e3c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:56.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- 172.21.15.49:0/3248819342 shutdown_connections 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/3248819342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4810ab70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- 172.21.15.49:0/3248819342 >> 172.21.15.49:0/3248819342 conn(0x7f8e481021b0 msgr2=0x7f8e481045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- 172.21.15.49:0/3248819342 shutdown_connections 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- 172.21.15.49:0/3248819342 wait complete. 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 Processor -- start 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.240+0000 7f8e4ec39700 1 -- start start 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4ec39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4ec39700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4810c110 con 0x7f8e48108780 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4c9d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4c9d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46990/0 (socket says 172.21.15.49:46990) 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4c9d5700 1 -- 172.21.15.49:0/2430380071 learned_addr learned my addr 172.21.15.49:0/2430380071 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4c9d5700 1 -- 172.21.15.49:0/2430380071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4819c0d0 con 0x7f8e48108780 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4c9d5700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0117e0 tx=0x7f8e3c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:56.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e3c003bf0 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3c01a910 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4819c360 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.241+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e3c024970 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.242+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4819c7c0 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.242+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f8e3c022020 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.242+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e48063900 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.243+0000 7f8e45ffb700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 0x7f8e3803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.243+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e3c04d1d0 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.243+0000 7f8e47fff700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 0x7f8e3803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.243+0000 7f8e47fff700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 0x7f8e3803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e34006fd0 tx=0x7f8e34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.247+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e3c011e20 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:55.510+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8e4819f250 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.296+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8e3c019c70 con 0x7f8e48108780 2024-03-16T08:07:56.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.299+0000 7f8e45ffb700 1 -- 172.21.15.49:0/2430380071 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f8e3c024cd0 con 0x7f8e48108780 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 msgr2=0x7f8e3803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 0x7f8e3803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e34006fd0 tx=0x7f8e34006e40 comp rx=0 tx=0).stop 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 msgr2=0x7f8e4819bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0117e0 tx=0x7f8e3c011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 shutdown_connections 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f8e38038ed0 0x7f8e3803b390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 --2- 172.21.15.49:0/2430380071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e48108780 0x7f8e4819bb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 >> 172.21.15.49:0/2430380071 conn(0x7f8e481021b0 msgr2=0x7f8e48102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 shutdown_connections 2024-03-16T08:07:56.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:56.300+0000 7f8e4ec39700 1 -- 172.21.15.49:0/2430380071 wait complete. 2024-03-16T08:07:57.286 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:56.897+0000 7fa0ad9b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:07:57.286 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:56.995+0000 7fa0ad9b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:07:57.549 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:57 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2430380071' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-16T08:07:57.549 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:57 smithi049 ceph-mon[31942]: mgrmap e9: x(active, since 19s) 2024-03-16T08:07:57.824 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:57.676+0000 7fa0ad9b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:07:57.824 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:57.772+0000 7fa0ad9b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:07:58.111 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:57.923+0000 7fa0ad9b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:07:58.111 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.023+0000 7fa0ad9b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:07:58.111 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.110+0000 7fa0ad9b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:07:58.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1975960665' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T08:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-16T08:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.548+0000 7f66862f1700 1 Processor -- start 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.548+0000 7f66862f1700 1 -- start start 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f66862f1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f66862f1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f668007be00 con 0x7f66801065f0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f667ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f667ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47024/0 (socket says 172.21.15.49:47024) 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f667ffff700 1 -- 172.21.15.49:0/3759878555 learned_addr learned my addr 172.21.15.49:0/3759878555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f667ffff700 1 -- 172.21.15.49:0/3759878555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680106f50 con 0x7f66801065f0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.549+0000 7f667ffff700 1 --2- 172.21.15.49:0/3759878555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66700098a0 tx=0x7f6670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5854eff77f5a653f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f667effd700 1 -- 172.21.15.49:0/3759878555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6670004160 con 0x7f66801065f0 2024-03-16T08:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f667effd700 1 -- 172.21.15.49:0/3759878555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6670004300 con 0x7f66801065f0 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f667effd700 1 -- 172.21.15.49:0/3759878555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6670011b20 con 0x7f66801065f0 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f66862f1700 1 -- 172.21.15.49:0/3759878555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 msgr2=0x7f6680106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f66862f1700 1 --2- 172.21.15.49:0/3759878555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66700098a0 tx=0x7f6670009bb0 comp rx=0 tx=0).stop 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f66862f1700 1 -- 172.21.15.49:0/3759878555 shutdown_connections 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f66862f1700 1 --2- 172.21.15.49:0/3759878555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f6680106a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.550+0000 7f66862f1700 1 -- 172.21.15.49:0/3759878555 >> 172.21.15.49:0/3759878555 conn(0x7f6680102230 msgr2=0x7f6680104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.551+0000 7f66862f1700 1 -- 172.21.15.49:0/3759878555 shutdown_connections 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.551+0000 7f66862f1700 1 -- 172.21.15.49:0/3759878555 wait complete. 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.551+0000 7f66862f1700 1 Processor -- start 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.551+0000 7f66862f1700 1 -- start start 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f66862f1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f66862f1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6680108020 con 0x7f66801065f0 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47040/0 (socket says 172.21.15.49:47040) 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667ffff700 1 -- 172.21.15.49:0/1975960665 learned_addr learned my addr 172.21.15.49:0/1975960665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667ffff700 1 -- 172.21.15.49:0/1975960665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668019e310 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667ffff700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66700117e0 tx=0x7f6670011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6670003bf0 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667001a910 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f66862f1700 1 -- 172.21.15.49:0/1975960665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668019e5a0 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6670024970 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.552+0000 7f66862f1700 1 -- 172.21.15.49:0/1975960665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668019ea00 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.553+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660005330 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.554+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f6670022020 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.554+0000 7f667d7fa700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 0x7f666803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.554+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f667004d1f0 con 0x7f66801065f0 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.554+0000 7f667f7fe700 1 -- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 msgr2=0x7f666803b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.554+0000 7f667f7fe700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 0x7f666803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.559+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6670011e20 con 0x7f66801065f0 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.755+0000 7f667f7fe700 1 -- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 msgr2=0x7f666803b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.755+0000 7f667f7fe700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 0x7f666803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.824+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6660005e30 con 0x7f66801065f0 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.824+0000 7f667d7fa700 1 -- 172.21.15.49:0/1975960665 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f667000b3e0 con 0x7f66801065f0 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.826+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 msgr2=0x7f666803b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.826+0000 7f6666ffd700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 0x7f666803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.826+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 msgr2=0x7f668019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.826+0000 7f6666ffd700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66700117e0 tx=0x7f6670011c10 comp rx=0 tx=0).stop 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 shutdown_connections 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7f6668038f80 0x7f666803b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 --2- 172.21.15.49:0/1975960665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66801065f0 0x7f668019ddd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 >> 172.21.15.49:0/1975960665 conn(0x7f6680102230 msgr2=0x7f6680194a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 shutdown_connections 2024-03-16T08:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:57.827+0000 7f6666ffd700 1 -- 172.21.15.49:0/1975960665 wait complete. 2024-03-16T08:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-16T08:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-03-16T08:07:58.694 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.387+0000 7fa0ad9b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:07:58.694 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.503+0000 7fa0ad9b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:07:58.694 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.594+0000 7fa0ad9b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:07:59.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.694+0000 7fa0ad9b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:07:59.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:58.881+0000 7fa0ad9b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:08:00.186 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:07:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:07:59.909+0000 7fa0ad9b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:08:00.186 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:00.049+0000 7fa0ad9b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:08:00.501 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:00.406+0000 7fa0ad9b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:08:00.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:00.501+0000 7fa0ad9b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:08:00.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:00.708+0000 7fa0ad9b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:08:01.204 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:00.905+0000 7fa0ad9b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:08:01.473 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:01.204+0000 7fa0ad9b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:08:01.473 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:01.473+0000 7fa0ad9b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:08:01.786 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:01.719+0000 7fa0ad9b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:08:03.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:02.560+0000 7fa0ad9b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:08:03.483 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:03.155+0000 7fa0ad9b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:08:03.483 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:03.258+0000 7fa0ad9b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:08:03.484 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:08:03.484+0000 7fa0ad9b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:08:03.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: Active manager daemon x restarted 2024-03-16T08:08:03.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: Activating manager daemon x 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: osdmap e3: 0 total, 0 up, 0 in 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: mgrmap e10: x(active, starting, since 0.00347635s) 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: Manager daemon x is now available 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:04.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: [16/Mar/2024:08:08:04] ENGINE Bus STARTING 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: [16/Mar/2024:08:08:04] ENGINE Serving on http://172.21.15.49:8765 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: [16/Mar/2024:08:08:04] ENGINE Serving on https://172.21.15.49:7150 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: [16/Mar/2024:08:08:04] ENGINE Bus STARTED 2024-03-16T08:08:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:04 smithi049 ceph-mon[31942]: mgrmap e11: x(active, since 1.00615s) 2024-03-16T08:08:05.444 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd88272e700 1 Processor -- start 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd88272e700 1 -- start start 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd88272e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd88272e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c07be00 con 0x7fd87c108800 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd87bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd87bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47048/0 (socket says 172.21.15.49:47048) 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.914+0000 7fd87bfff700 1 -- 172.21.15.49:0/2429969540 learned_addr learned my addr 172.21.15.49:0/2429969540 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.915+0000 7fd87bfff700 1 -- 172.21.15.49:0/2429969540 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c10b130 con 0x7fd87c108800 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.915+0000 7fd87bfff700 1 --2- 172.21.15.49:0/2429969540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0098a0 tx=0x7fd86c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b4323123c632d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:05.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.915+0000 7fd87affd700 1 -- 172.21.15.49:0/2429969540 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd86c004160 con 0x7fd87c108800 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.915+0000 7fd87affd700 1 -- 172.21.15.49:0/2429969540 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c004300 con 0x7fd87c108800 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.915+0000 7fd87affd700 1 -- 172.21.15.49:0/2429969540 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd86c011b20 con 0x7fd87c108800 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- 172.21.15.49:0/2429969540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 msgr2=0x7fd87c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 --2- 172.21.15.49:0/2429969540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0098a0 tx=0x7fd86c009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- 172.21.15.49:0/2429969540 shutdown_connections 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 --2- 172.21.15.49:0/2429969540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c10abf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- 172.21.15.49:0/2429969540 >> 172.21.15.49:0/2429969540 conn(0x7fd87c102230 msgr2=0x7fd87c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- 172.21.15.49:0/2429969540 shutdown_connections 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- 172.21.15.49:0/2429969540 wait complete. 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 Processor -- start 2024-03-16T08:08:05.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.916+0000 7fd88272e700 1 -- start start 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd88272e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd88272e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10c190 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd87bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd87bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47062/0 (socket says 172.21.15.49:47062) 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd87bfff700 1 -- 172.21.15.49:0/1850565793 learned_addr learned my addr 172.21.15.49:0/1850565793 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd87bfff700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c19c0c0 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd87bfff700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0117e0 tx=0x7fd86c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd86c003bf0 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.917+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c01a910 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.918+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd86c024970 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.918+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c19c350 con 0x7fd87c108800 2024-03-16T08:08:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.918+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c19c7b0 con 0x7fd87c108800 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.918+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7fd86c022020 con 0x7fd87c108800 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.919+0000 7fd8797fa700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.919+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd87c0693a0 con 0x7fd864038f20 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.919+0000 7fd87b7fe700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.919+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd86c04d330 con 0x7fd87c108800 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:58.919+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:59.120+0000 7fd87b7fe700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:59.120+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:59.521+0000 7fd87b7fe700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:07:59.521+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:00.322+0000 7fd87b7fe700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:00.322+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T08:08:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:01.923+0000 7fd87b7fe700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2659026035 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:01.923+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:03.500+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7fd86c01aac0 con 0x7fd87c108800 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:03.500+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 msgr2=0x7fd86403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:03.500+0000 7fd8797fa700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.501+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7fd86c04ef60 con 0x7fd87c108800 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.501+0000 7fd8797fa700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 0x7fd86403eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.502+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd86c00be90 con 0x7fd86403c7b0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.502+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 0x7fd86403eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.503+0000 7fd87b7fe700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 0x7fd86403eba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd870003a10 tx=0x7fd8700092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.503+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fd86c00be90 con 0x7fd86403c7b0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd87c103390 con 0x7fd86403c7b0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd8797fa700 1 -- 172.21.15.49:0/1850565793 <== mgr.14152 v2:172.21.15.49:6800/2256169095 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd87c103390 con 0x7fd86403c7b0 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 msgr2=0x7fd86403eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 0x7fd86403eba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd870003a10 tx=0x7fd8700092b0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 msgr2=0x7fd87c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0117e0 tx=0x7fd86c011c10 comp rx=0 tx=0).stop 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 shutdown_connections 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.510+0000 7fd88272e700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd86403c7b0 0x7fd86403eba0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.511+0000 7fd88272e700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd87c108800 0x7fd87c19bb80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.511+0000 7fd88272e700 1 --2- 172.21.15.49:0/1850565793 >> [v2:172.21.15.49:6800/2659026035,v1:172.21.15.49:6801/2659026035] conn(0x7fd864038f20 0x7fd86403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.511+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 >> 172.21.15.49:0/1850565793 conn(0x7fd87c102230 msgr2=0x7fd87c102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.511+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 shutdown_connections 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:04.511+0000 7fd88272e700 1 -- 172.21.15.49:0/1850565793 wait complete. 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-03-16T08:08:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2024-03-16T08:08:05.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:05 smithi049 ceph-mon[31942]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T08:08:05.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:05 smithi049 ceph-mon[31942]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T08:08:05.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:06.715 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.849+0000 7f1c772a2700 1 Processor -- start 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.849+0000 7f1c772a2700 1 -- start start 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c772a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c772a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7007be00 con 0x7f1c70108800 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c7503e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c7503e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34828/0 (socket says 172.21.15.49:34828) 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c7503e700 1 -- 172.21.15.49:0/1772355029 learned_addr learned my addr 172.21.15.49:0/1772355029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c7503e700 1 -- 172.21.15.49:0/1772355029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7010b130 con 0x7f1c70108800 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c7503e700 1 --2- 172.21.15.49:0/1772355029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c640098a0 tx=0x7f1c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5876188500b086c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c63fff700 1 -- 172.21.15.49:0/1772355029 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c64004160 con 0x7f1c70108800 2024-03-16T08:08:06.716 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.850+0000 7f1c63fff700 1 -- 172.21.15.49:0/1772355029 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c64004300 con 0x7f1c70108800 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c63fff700 1 -- 172.21.15.49:0/1772355029 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c64011b20 con 0x7f1c70108800 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 -- 172.21.15.49:0/1772355029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 msgr2=0x7f1c7010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 --2- 172.21.15.49:0/1772355029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c640098a0 tx=0x7f1c64009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 -- 172.21.15.49:0/1772355029 shutdown_connections 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 --2- 172.21.15.49:0/1772355029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c7010abf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 -- 172.21.15.49:0/1772355029 >> 172.21.15.49:0/1772355029 conn(0x7f1c70102230 msgr2=0x7f1c70104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 -- 172.21.15.49:0/1772355029 shutdown_connections 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.851+0000 7f1c772a2700 1 -- 172.21.15.49:0/1772355029 wait complete. 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c772a2700 1 Processor -- start 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c772a2700 1 -- start start 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c772a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c772a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010c190 con 0x7f1c70108800 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c7503e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c7503e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34842/0 (socket says 172.21.15.49:34842) 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.852+0000 7f1c7503e700 1 -- 172.21.15.49:0/4098823165 learned_addr learned my addr 172.21.15.49:0/4098823165 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:06.717 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c7503e700 1 -- 172.21.15.49:0/4098823165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c701085c0 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c7503e700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c640117e0 tx=0x7f1c64011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c64003bf0 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6401a910 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c701067b0 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c64024970 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.853+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c70106c70 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.854+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f1c64022020 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.854+0000 7f1c627fc700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 0x7f1c5c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.854+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c700693a0 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.854+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c6404d3b0 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.854+0000 7f1c7483d700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 0x7f1c5c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.855+0000 7f1c7483d700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 0x7f1c5c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c006fd0 tx=0x7f1c6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:05.859+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1c64011e20 con 0x7f1c70108800 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.072+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1c70107550 con 0x7f1c5c038f20 2024-03-16T08:08:06.718 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.149+0000 7f1c627fc700 1 -- 172.21.15.49:0/4098823165 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1c70107550 con 0x7f1c5c038f20 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 msgr2=0x7f1c5c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 0x7f1c5c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c006fd0 tx=0x7f1c6c006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 msgr2=0x7f1c70108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c640117e0 tx=0x7f1c64011c10 comp rx=0 tx=0).stop 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 shutdown_connections 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c038f20 0x7f1c5c03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 --2- 172.21.15.49:0/4098823165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70108800 0x7f1c70108080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 >> 172.21.15.49:0/4098823165 conn(0x7f1c70102230 msgr2=0x7f1c70102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 shutdown_connections 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:06.151+0000 7f1c772a2700 1 -- 172.21.15.49:0/4098823165 wait complete. 2024-03-16T08:08:06.719 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-03-16T08:08:06.833 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:06 smithi049 ceph-mon[31942]: mgrmap e12: x(active, since 2s) 2024-03-16T08:08:06.833 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:06 smithi049 ceph-mon[31942]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:06.834 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:06.834 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:07.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fj6NkVwSP7XTEo51BYr3qufTRxn2D4EUaMQCyvTy6fq7nPB8Z2wVm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710576487, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8c94c700 1 Processor -- start 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8c94c700 1 -- start start 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8c94c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8c94c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec881086c0 con 0x7fec88107cd0 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.165+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34848/0 (socket says 172.21.15.49:34848) 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8659c700 1 -- 172.21.15.49:0/3213238699 learned_addr learned my addr 172.21.15.49:0/3213238699 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8659c700 1 -- 172.21.15.49:0/3213238699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec88108e80 con 0x7fec88107cd0 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8659c700 1 --2- 172.21.15.49:0/3213238699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec700098a0 tx=0x7fec70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b075578a0db9ae15 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8559a700 1 -- 172.21.15.49:0/3213238699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec70004160 con 0x7fec88107cd0 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8559a700 1 -- 172.21.15.49:0/3213238699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec70004300 con 0x7fec88107cd0 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.166+0000 7fec8559a700 1 -- 172.21.15.49:0/3213238699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec70011b20 con 0x7fec88107cd0 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 -- 172.21.15.49:0/3213238699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 msgr2=0x7fec881080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 --2- 172.21.15.49:0/3213238699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec700098a0 tx=0x7fec70009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 -- 172.21.15.49:0/3213238699 shutdown_connections 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 --2- 172.21.15.49:0/3213238699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec881080f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 -- 172.21.15.49:0/3213238699 >> 172.21.15.49:0/3213238699 conn(0x7fec881032b0 msgr2=0x7fec881056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 -- 172.21.15.49:0/3213238699 shutdown_connections 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 -- 172.21.15.49:0/3213238699 wait complete. 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.167+0000 7fec8c94c700 1 Processor -- start 2024-03-16T08:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8c94c700 1 -- start start 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8c94c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8c94c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec8810c230 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34854/0 (socket says 172.21.15.49:34854) 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8659c700 1 -- 172.21.15.49:0/2390932189 learned_addr learned my addr 172.21.15.49:0/2390932189 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8659c700 1 -- 172.21.15.49:0/2390932189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8807e040 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8659c700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec700117e0 tx=0x7fec70011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec70003bf0 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec7001a910 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.168+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8807c230 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.169+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec70024970 con 0x7fec88107cd0 2024-03-16T08:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.169+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8807c6f0 con 0x7fec88107cd0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.169+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fec70022020 con 0x7fec88107cd0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.169+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec880693a0 con 0x7fec88107cd0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.170+0000 7fec7f7fe700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 0x7fec7403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.170+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec7004d980 con 0x7fec88107cd0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.170+0000 7fec85d9b700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 0x7fec7403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.170+0000 7fec85d9b700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 0x7fec7403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec78006fd0 tx=0x7fec78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.175+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec7002cdf0 con 0x7fec88107cd0 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.405+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fec8807ce60 con 0x7fec74038f70 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.653+0000 7fec7f7fe700 1 -- 172.21.15.49:0/2390932189 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fec8807ce60 con 0x7fec74038f70 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 msgr2=0x7fec7403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 0x7fec7403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec78006fd0 tx=0x7fec78006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 msgr2=0x7fec8807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec700117e0 tx=0x7fec70011ba0 comp rx=0 tx=0).stop 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 shutdown_connections 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec74038f70 0x7fec7403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 --2- 172.21.15.49:0/2390932189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88107cd0 0x7fec8807db00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.655+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 >> 172.21.15.49:0/2390932189 conn(0x7fec881032b0 msgr2=0x7fec88103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.656+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 shutdown_connections 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:07.656+0000 7fec8c94c700 1 -- 172.21.15.49:0/2390932189 wait complete. 2024-03-16T08:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-03-16T08:08:08.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:08 smithi049 ceph-mon[31942]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:08.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:09.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:09 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4293800148' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-16T08:08:09.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-03-16T08:08:09.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36fea18700 1 Processor -- start 2024-03-16T08:08:09.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36fea18700 1 -- start start 2024-03-16T08:08:09.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36fea18700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:09.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36fea18700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36f81075c0 con 0x7f36f8106bd0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34870/0 (socket says 172.21.15.49:34870) 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.773+0000 7f36f7fff700 1 -- 172.21.15.49:0/1531090807 learned_addr learned my addr 172.21.15.49:0/1531090807 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36f7fff700 1 -- 172.21.15.49:0/1531090807 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f8107dd0 con 0x7f36f8106bd0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36f7fff700 1 --2- 172.21.15.49:0/1531090807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098a0 tx=0x7f36e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d7da915b3eae20 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36f6ffd700 1 -- 172.21.15.49:0/1531090807 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e8004160 con 0x7f36f8106bd0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36f6ffd700 1 -- 172.21.15.49:0/1531090807 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e8004300 con 0x7f36f8106bd0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36f6ffd700 1 -- 172.21.15.49:0/1531090807 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e8011b20 con 0x7f36f8106bd0 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36fea18700 1 -- 172.21.15.49:0/1531090807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 msgr2=0x7f36f8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36fea18700 1 --2- 172.21.15.49:0/1531090807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098a0 tx=0x7f36e8009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36fea18700 1 -- 172.21.15.49:0/1531090807 shutdown_connections 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36fea18700 1 --2- 172.21.15.49:0/1531090807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f8106ff0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.774+0000 7f36fea18700 1 -- 172.21.15.49:0/1531090807 >> 172.21.15.49:0/1531090807 conn(0x7f36f81021b0 msgr2=0x7f36f81045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.775+0000 7f36fea18700 1 -- 172.21.15.49:0/1531090807 shutdown_connections 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.775+0000 7f36fea18700 1 -- 172.21.15.49:0/1531090807 wait complete. 2024-03-16T08:08:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.775+0000 7f36fea18700 1 Processor -- start 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.775+0000 7f36fea18700 1 -- start start 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36fea18700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36fea18700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36f810b180 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34872/0 (socket says 172.21.15.49:34872) 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f7fff700 1 -- 172.21.15.49:0/4293800148 learned_addr learned my addr 172.21.15.49:0/4293800148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f7fff700 1 -- 172.21.15.49:0/4293800148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f819c180 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f7fff700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36e80117e0 tx=0x7f36e8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e8003bf0 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e801a910 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f819c410 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e8024970 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.776+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f819c870 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.777+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f36e8022020 con 0x7f36f8106bd0 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.777+0000 7f36f57fa700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 0x7f36e003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:09.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.777+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f8063900 con 0x7f36f8106bd0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.778+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f36e804d720 con 0x7f36f8106bd0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.778+0000 7f36f77fe700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 0x7f36e003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.778+0000 7f36f77fe700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 0x7f36e003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36ec006fd0 tx=0x7f36ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.783+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f36e80172b0 con 0x7f36f8106bd0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.991+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f36f8062ee0 con 0x7f36f8106bd0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.992+0000 7f36f57fa700 1 -- 172.21.15.49:0/4293800148 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f36e800b5a0 con 0x7f36f8106bd0 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 msgr2=0x7f36e003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 0x7f36e003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36ec006fd0 tx=0x7f36ec006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 msgr2=0x7f36f819bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36e80117e0 tx=0x7f36e8011ba0 comp rx=0 tx=0).stop 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 shutdown_connections 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36e0038f70 0x7f36e003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 --2- 172.21.15.49:0/4293800148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f8106bd0 0x7f36f819bc40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:09.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 >> 172.21.15.49:0/4293800148 conn(0x7f36f81021b0 msgr2=0x7f36f81927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:09.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 shutdown_connections 2024-03-16T08:08:09.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:08.994+0000 7f36fea18700 1 -- 172.21.15.49:0/4293800148 wait complete. 2024-03-16T08:08:09.753 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:08:09.753 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-16T08:08:09.761 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:08:09.761 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-16T08:08:09.761 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-16T08:08:09.761 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: Password: tt9epzp5er 2024-03-16T08:08:09.763 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:09.764 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config directory 2024-03-16T08:08:09.764 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2024-03-16T08:08:10.883 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff2d494700 1 Processor -- start 2024-03-16T08:08:10.883 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff2d494700 1 -- start start 2024-03-16T08:08:10.883 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff2d494700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:10.883 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff2d494700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff2807c860 con 0x7eff2807dde0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34882/0 (socket says 172.21.15.49:34882) 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff26ffd700 1 -- 172.21.15.49:0/4035657465 learned_addr learned my addr 172.21.15.49:0/4035657465 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.173+0000 7eff26ffd700 1 -- 172.21.15.49:0/4035657465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2807c9e0 con 0x7eff2807dde0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff26ffd700 1 --2- 172.21.15.49:0/4035657465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff180098a0 tx=0x7eff18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bbbd25ce11341c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff25ffb700 1 -- 172.21.15.49:0/4035657465 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18004160 con 0x7eff2807dde0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff25ffb700 1 -- 172.21.15.49:0/4035657465 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff18004300 con 0x7eff2807dde0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff25ffb700 1 -- 172.21.15.49:0/4035657465 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18011b20 con 0x7eff2807dde0 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 -- 172.21.15.49:0/4035657465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 msgr2=0x7eff2807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 --2- 172.21.15.49:0/4035657465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff180098a0 tx=0x7eff18009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 -- 172.21.15.49:0/4035657465 shutdown_connections 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 --2- 172.21.15.49:0/4035657465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2807c290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 -- 172.21.15.49:0/4035657465 >> 172.21.15.49:0/4035657465 conn(0x7eff28103270 msgr2=0x7eff281056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 -- 172.21.15.49:0/4035657465 shutdown_connections 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.174+0000 7eff2d494700 1 -- 172.21.15.49:0/4035657465 wait complete. 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff2d494700 1 Processor -- start 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff2d494700 1 -- start start 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff2d494700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff2d494700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810e2d0 con 0x7eff2807dde0 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34892/0 (socket says 172.21.15.49:34892) 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.175+0000 7eff26ffd700 1 -- 172.21.15.49:0/2253817562 learned_addr learned my addr 172.21.15.49:0/2253817562 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff26ffd700 1 -- 172.21.15.49:0/2253817562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2819e2f0 con 0x7eff2807dde0 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff26ffd700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff180116a0 tx=0x7eff18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:10.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18003a40 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2819e580 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff18003be0 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff1801a750 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.176+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281a1230 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.177+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7eff1801aa90 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.177+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff28063900 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.177+0000 7eff0ffff700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 0x7eff1003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.177+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff1804d7e0 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.177+0000 7eff267fc700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 0x7eff1003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.178+0000 7eff267fc700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 0x7eff1003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006fd0 tx=0x7eff1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.182+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff18011e20 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.390+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7eff2819e9e0 con 0x7eff2807dde0 2024-03-16T08:08:10.886 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.391+0000 7eff0ffff700 1 -- 172.21.15.49:0/2253817562 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7eff1802d910 con 0x7eff2807dde0 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 msgr2=0x7eff1003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 0x7eff1003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006fd0 tx=0x7eff1c006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 msgr2=0x7eff2819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff180116a0 tx=0x7eff18011ba0 comp rx=0 tx=0).stop 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 shutdown_connections 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff10038fc0 0x7eff1003b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 --2- 172.21.15.49:0/2253817562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff2807dde0 0x7eff2819ddb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 >> 172.21.15.49:0/2253817562 conn(0x7eff28103270 msgr2=0x7eff28103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 shutdown_connections 2024-03-16T08:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:10.393+0000 7eff2d494700 1 -- 172.21.15.49:0/2253817562 wait complete. 2024-03-16T08:08:10.992 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:10 smithi049 ceph-mon[31942]: mgrmap e13: x(active, since 6s) 2024-03-16T08:08:12.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.336+0000 7f82194ea700 1 Processor -- start 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.336+0000 7f82194ea700 1 -- start start 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f82194ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f82194ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82141086e0 con 0x7f8214107cf0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34908/0 (socket says 172.21.15.49:34908) 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8212ffd700 1 -- 172.21.15.49:0/1258467223 learned_addr learned my addr 172.21.15.49:0/1258467223 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8212ffd700 1 -- 172.21.15.49:0/1258467223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8214108ea0 con 0x7f8214107cf0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8212ffd700 1 --2- 172.21.15.49:0/1258467223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82040098a0 tx=0x7f8204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d369dc4ec859852f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8211ffb700 1 -- 172.21.15.49:0/1258467223 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8204004160 con 0x7f8214107cf0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8211ffb700 1 -- 172.21.15.49:0/1258467223 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8204004300 con 0x7f8214107cf0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.337+0000 7f8211ffb700 1 -- 172.21.15.49:0/1258467223 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8204011b20 con 0x7f8214107cf0 2024-03-16T08:08:12.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 -- 172.21.15.49:0/1258467223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 msgr2=0x7f8214108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 --2- 172.21.15.49:0/1258467223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82040098a0 tx=0x7f8204009bb0 comp rx=0 tx=0).stop 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 -- 172.21.15.49:0/1258467223 shutdown_connections 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 --2- 172.21.15.49:0/1258467223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f8214108110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 -- 172.21.15.49:0/1258467223 >> 172.21.15.49:0/1258467223 conn(0x7f8214103270 msgr2=0x7f82141056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 -- 172.21.15.49:0/1258467223 shutdown_connections 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.338+0000 7f82194ea700 1 -- 172.21.15.49:0/1258467223 wait complete. 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f82194ea700 1 Processor -- start 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f82194ea700 1 -- start start 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f82194ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f82194ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f821410c250 con 0x7f8214107cf0 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34914/0 (socket says 172.21.15.49:34914) 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f8212ffd700 1 -- 172.21.15.49:0/4237354169 learned_addr learned my addr 172.21.15.49:0/4237354169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.339+0000 7f8212ffd700 1 -- 172.21.15.49:0/4237354169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821407e040 con 0x7f8214107cf0 2024-03-16T08:08:12.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f8212ffd700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82040116a0 tx=0x7f8204011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8204003a40 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8204003be0 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821407c230 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f820401a720 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.340+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821407c6f0 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f820401a9f0 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8214063900 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f81fbfff700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 0x7f81fc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f820404d700 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f82127fc700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 0x7f81fc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.341+0000 7f82127fc700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 0x7f81fc03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8208006fd0 tx=0x7f8208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.346+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8204011e20 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.664+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f821407ca30 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.666+0000 7f81fbfff700 1 -- 172.21.15.49:0/4237354169 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f821407ca30 con 0x7f8214107cf0 2024-03-16T08:08:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.668+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 msgr2=0x7f81fc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.668+0000 7f82194ea700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 0x7f81fc03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8208006fd0 tx=0x7f8208006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.668+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 msgr2=0x7f821407db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.668+0000 7f82194ea700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82040116a0 tx=0x7f8204011ba0 comp rx=0 tx=0).stop 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 shutdown_connections 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81fc039020 0x7f81fc03b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 --2- 172.21.15.49:0/4237354169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8214107cf0 0x7f821407db00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 >> 172.21.15.49:0/4237354169 conn(0x7f8214103270 msgr2=0x7f8214103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 shutdown_connections 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-16T08:08:11.669+0000 7f82194ea700 1 -- 172.21.15.49:0/4237354169 wait complete. 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-03-16T08:08:12.193 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:12.194 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-03-16T08:08:12.234 INFO:tasks.cephadm:Fetching config... 2024-03-16T08:08:12.234 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:08:12.235 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-16T08:08:12.251 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-16T08:08:12.251 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:08:12.251 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-16T08:08:12.306 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-16T08:08:12.306 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:08:12.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-16T08:08:12.377 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-16T08:08:12.377 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:08:12.377 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-16T08:08:12.433 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-16T08:08:12.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T08:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:12.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T08:08:12.584 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:12.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T08:08:12.647 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+UZyVaatIKVNe9D1cPYWnLd/P9FuIs5c3CUZeh3hzrfAVnS68fgjdmC8kbmMabKks1pFxSgloeO7z1z2SERVDZ2ZnfRGNVsz970flCvJ3Hdhxcu4yrWKwLGhWwNbdAetiwNwJOy3QJXGMf9+uiZjM1dZNxq/KYfeHUW0lAYsQ4S80pD0l5MHk+aehWgIBa5DOFQeK2mnp37+CvwZp/7RpKgFDZ431n0RS3J9LMcsel1rBBu4ZOAyd8nARWwyZzre2HQiVi+v0lGeIEgM6r87YzE2K9S/ATMmEjZn4025M24xPqXpV4sxbuu1Xd+IN5y0aMPAv17B574+119Us/eFfj9uMbZm0AnjAfVHgR80HJhyeg9WAdvTAo8UZ8WSWvUstK57BS5kRfuDYtSuzbgfypzGwo4AZ1bZnzgHX62IJmh0cdMww/BQ6wPj2ffzlJCGmnqVeXF6yb9dqQ5QbU37pk+11Jjnc6EMcQyPj/t00yhJM7EFhpMHhbpV0SGUYX2U= ceph-053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:12.662 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-16T08:08:12.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:12 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4237354169' entity='client.admin' 2024-03-16T08:08:12.909 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 -- 172.21.15.49:0/437519382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 msgr2=0x7f3a38102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/437519382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38102140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009910 tx=0x7f3a28009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 -- 172.21.15.49:0/437519382 shutdown_connections 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/437519382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38102140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 -- 172.21.15.49:0/437519382 >> 172.21.15.49:0/437519382 conn(0x7f3a380fd2c0 msgr2=0x7f3a380ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 -- 172.21.15.49:0/437519382 shutdown_connections 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.099+0000 7f3a3f25b700 1 -- 172.21.15.49:0/437519382 wait complete. 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3f25b700 1 Processor -- start 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3f25b700 1 -- start start 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3f25b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3f25b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38107db0 con 0x7f3a38101d20 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3cff7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3cff7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33874/0 (socket says 172.21.15.49:33874) 2024-03-16T08:08:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3cff7700 1 -- 172.21.15.49:0/2772451382 learned_addr learned my addr 172.21.15.49:0/2772451382 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3cff7700 1 -- 172.21.15.49:0/2772451382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a38197990 con 0x7f3a38101d20 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.100+0000 7f3a3cff7700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a28005850 tx=0x7f3a28005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.101+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a28026070 con 0x7f3a38101d20 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.101+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a280044a0 con 0x7f3a38101d20 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.101+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a38197c20 con 0x7f3a38101d20 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.101+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a28003ab0 con 0x7f3a38101d20 2024-03-16T08:08:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.101+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3819a910 con 0x7f3a38101d20 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3a28029070 con 0x7f3a38101d20 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a38108530 con 0x7f3a38101d20 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a35ffb700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 0x7f3a2003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a2801e070 con 0x7f3a38101d20 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a37fff700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 0x7f3a2003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.102+0000 7f3a37fff700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 0x7f3a2003b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006fd0 tx=0x7f3a2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.107+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a28013070 con 0x7f3a38101d20 2024-03-16T08:08:15.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.316+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3a3802dc80 con 0x7f3a38101d20 2024-03-16T08:08:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.319+0000 7f3a35ffb700 1 -- 172.21.15.49:0/2772451382 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3a28030070 con 0x7f3a38101d20 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 msgr2=0x7f3a2003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 0x7f3a2003b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006fd0 tx=0x7f3a2c006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 msgr2=0x7f3a38197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a28005850 tx=0x7f3a28005030 comp rx=0 tx=0).stop 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 shutdown_connections 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f3a20039020 0x7f3a2003b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 --2- 172.21.15.49:0/2772451382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a38101d20 0x7f3a38197450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 >> 172.21.15.49:0/2772451382 conn(0x7f3a380fd2c0 msgr2=0x7f3a380fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 shutdown_connections 2024-03-16T08:08:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:15.321+0000 7f3a3f25b700 1 -- 172.21.15.49:0/2772451382 wait complete. 2024-03-16T08:08:15.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:15.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2772451382' entity='client.admin' 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:15.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:16.052 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-16T08:08:16.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-16T08:08:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.265+0000 7feff2f07700 1 -- 172.21.15.49:0/346225339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 msgr2=0x7fefec0746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.265+0000 7feff2f07700 1 --2- 172.21.15.49:0/346225339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec0746e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009910 tx=0x7fefe0009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.265+0000 7feff2f07700 1 -- 172.21.15.49:0/346225339 shutdown_connections 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.265+0000 7feff2f07700 1 --2- 172.21.15.49:0/346225339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec0746e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.265+0000 7feff2f07700 1 -- 172.21.15.49:0/346225339 >> 172.21.15.49:0/346225339 conn(0x7fefec0fd2c0 msgr2=0x7fefec0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 -- 172.21.15.49:0/346225339 shutdown_connections 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 -- 172.21.15.49:0/346225339 wait complete. 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 Processor -- start 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 -- start start 2024-03-16T08:08:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff2f07700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefec109fa0 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff0ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff0ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33906/0 (socket says 172.21.15.49:33906) 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.266+0000 7feff0ca3700 1 -- 172.21.15.49:0/2681573774 learned_addr learned my addr 172.21.15.49:0/2681573774 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7feff0ca3700 1 -- 172.21.15.49:0/2681573774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec199b60 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7feff0ca3700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefe0004fa0 tx=0x7fefe0005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0027070 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe000be40 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec199df0 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe000f460 con 0x7fefec076280 2024-03-16T08:08:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.267+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec19cae0 con 0x7fefec076280 2024-03-16T08:08:18.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.268+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fefe002a070 con 0x7fefec076280 2024-03-16T08:08:18.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.268+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec10a720 con 0x7fefec076280 2024-03-16T08:08:18.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.268+0000 7fefe9ffb700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 0x7fefdc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:18.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.268+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefe001e030 con 0x7fefec076280 2024-03-16T08:08:18.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.269+0000 7fefebfff700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 0x7fefdc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:18.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.269+0000 7fefebfff700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 0x7fefdc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefd8006fd0 tx=0x7fefd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:18.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.273+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fefe0013070 con 0x7fefec076280 2024-03-16T08:08:18.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.483+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fefec02dc80 con 0x7fefdc039020 2024-03-16T08:08:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.486+0000 7fefe9ffb700 1 -- 172.21.15.49:0/2681573774 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fefec02dc80 con 0x7fefdc039020 2024-03-16T08:08:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 msgr2=0x7fefdc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 0x7fefdc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefd8006fd0 tx=0x7fefd8006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 msgr2=0x7fefec199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefe0004fa0 tx=0x7fefe0005080 comp rx=0 tx=0).stop 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 shutdown_connections 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fefdc039020 0x7fefdc03b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 --2- 172.21.15.49:0/2681573774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec076280 0x7fefec199620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 >> 172.21.15.49:0/2681573774 conn(0x7fefec0fd2c0 msgr2=0x7fefec0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 shutdown_connections 2024-03-16T08:08:18.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:18.488+0000 7feff2f07700 1 -- 172.21.15.49:0/2681573774 wait complete. 2024-03-16T08:08:19.257 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-03-16T08:08:19.257 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:08:19.257 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-03-16T08:08:19.276 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:08:19.276 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:19.333 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-03-16T08:08:19.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch host add smithi129 2024-03-16T08:08:19.582 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:19.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:19.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 -- 172.21.15.49:0/1135351461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 msgr2=0x7fca041011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 --2- 172.21.15.49:0/1135351461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca041011e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009910 tx=0x7fc9f4009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 -- 172.21.15.49:0/1135351461 shutdown_connections 2024-03-16T08:08:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 --2- 172.21.15.49:0/1135351461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca041011e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 -- 172.21.15.49:0/1135351461 >> 172.21.15.49:0/1135351461 conn(0x7fca040fc690 msgr2=0x7fca040feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 -- 172.21.15.49:0/1135351461 shutdown_connections 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.992+0000 7fca0a059700 1 -- 172.21.15.49:0/1135351461 wait complete. 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca0a059700 1 Processor -- start 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca0a059700 1 -- start start 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca0a059700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca0a059700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410b1d0 con 0x7fca04100e00 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33910/0 (socket says 172.21.15.49:33910) 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.993+0000 7fca037fe700 1 -- 172.21.15.49:0/2109464733 learned_addr learned my addr 172.21.15.49:0/2109464733 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca037fe700 1 -- 172.21.15.49:0/2109464733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0419d960 con 0x7fca04100e00 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca037fe700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4006010 tx=0x7fc9f4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4026070 con 0x7fca04100e00 2024-03-16T08:08:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f400bbd0 con 0x7fca04100e00 2024-03-16T08:08:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4003910 con 0x7fca04100e00 2024-03-16T08:08:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0419dc50 con 0x7fca04100e00 2024-03-16T08:08:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.994+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca041975e0 con 0x7fca04100e00 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.995+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fc9f4029070 con 0x7fca04100e00 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.995+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca041022e0 con 0x7fca04100e00 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.995+0000 7fca00ff9700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 0x7fc9ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.995+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9f401e070 con 0x7fca04100e00 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.995+0000 7fca02ffd700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 0x7fc9ec03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:20.996+0000 7fca02ffd700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 0x7fc9ec03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8006fd0 tx=0x7fc9f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:21.000+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc9f4013070 con 0x7fca04100e00 2024-03-16T08:08:21.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:21.212+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7fca04198b10 con 0x7fc9ec038fc0 2024-03-16T08:08:21.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:21 smithi049 ceph-mon[31942]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:22.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:22 smithi049 ceph-mon[31942]: Deploying cephadm binary to smithi129 2024-03-16T08:08:24.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:24 smithi049 ceph-mon[31942]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:25.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.524+0000 7fca00ff9700 1 -- 172.21.15.49:0/2109464733 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fca04198b10 con 0x7fc9ec038fc0 2024-03-16T08:08:25.526 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 msgr2=0x7fc9ec03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 0x7fc9ec03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8006fd0 tx=0x7fc9f8006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 msgr2=0x7fca0419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4006010 tx=0x7fc9f4005030 comp rx=0 tx=0).stop 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 shutdown_connections 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc9ec038fc0 0x7fc9ec03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 --2- 172.21.15.49:0/2109464733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca04100e00 0x7fca0419d420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.527+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 >> 172.21.15.49:0/2109464733 conn(0x7fca040fc690 msgr2=0x7fca0406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.528+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 shutdown_connections 2024-03-16T08:08:25.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:25.528+0000 7fca0a059700 1 -- 172.21.15.49:0/2109464733 wait complete. 2024-03-16T08:08:26.090 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T08:08:26.340 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:26.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:26 smithi049 ceph-mon[31942]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:26.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:26.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:26 smithi049 ceph-mon[31942]: Added host smithi129 2024-03-16T08:08:26.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:26.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:27.664 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:27.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 -- 172.21.15.49:0/4162440899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 msgr2=0x7f6e541011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:27.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/4162440899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541011e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009910 tx=0x7f6e50009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:27.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 -- 172.21.15.49:0/4162440899 shutdown_connections 2024-03-16T08:08:27.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/4162440899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541011e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 -- 172.21.15.49:0/4162440899 >> 172.21.15.49:0/4162440899 conn(0x7f6e540fc670 msgr2=0x7f6e540fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 -- 172.21.15.49:0/4162440899 shutdown_connections 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 -- 172.21.15.49:0/4162440899 wait complete. 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.871+0000 7f6e5b57d700 1 Processor -- start 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e5b57d700 1 -- start start 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e5b57d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e5b57d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5410b1d0 con 0x7f6e54100e00 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e59319700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e59319700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51852/0 (socket says 172.21.15.49:51852) 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e59319700 1 -- 172.21.15.49:0/585401738 learned_addr learned my addr 172.21.15.49:0/585401738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e59319700 1 -- 172.21.15.49:0/585401738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e54104790 con 0x7f6e54100e00 2024-03-16T08:08:27.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e59319700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006010 tx=0x7f6e50005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e50026070 con 0x7f6e54100e00 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5000bc30 con 0x7f6e54100e00 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e54102e40 con 0x7f6e54100e00 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.872+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e500039d0 con 0x7f6e54100e00 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.873+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e54103300 con 0x7f6e54100e00 2024-03-16T08:08:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.873+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f6e50029070 con 0x7f6e54100e00 2024-03-16T08:08:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.873+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e541022e0 con 0x7f6e54100e00 2024-03-16T08:08:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.874+0000 7f6e4a7fc700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 0x7f6e4003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.874+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e5001e070 con 0x7f6e54100e00 2024-03-16T08:08:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.874+0000 7f6e58b18700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 0x7f6e4003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.874+0000 7f6e58b18700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 0x7f6e4003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006fd0 tx=0x7f6e44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:27.879+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e50013070 con 0x7f6e54100e00 2024-03-16T08:08:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.090+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e54103dd0 con 0x7f6e40039020 2024-03-16T08:08:28.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.091+0000 7f6e4a7fc700 1 -- 172.21.15.49:0/585401738 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6e54103dd0 con 0x7f6e40039020 2024-03-16T08:08:28.092 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:28.092 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-03-16T08:08:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 msgr2=0x7f6e4003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 0x7f6e4003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006fd0 tx=0x7f6e44006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 msgr2=0x7f6e541028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006010 tx=0x7f6e50005030 comp rx=0 tx=0).stop 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 shutdown_connections 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6e40039020 0x7f6e4003b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 --2- 172.21.15.49:0/585401738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e54100e00 0x7f6e541028a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 >> 172.21.15.49:0/585401738 conn(0x7f6e540fc670 msgr2=0x7f6e5406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 shutdown_connections 2024-03-16T08:08:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:28.094+0000 7f6e5b57d700 1 -- 172.21.15.49:0/585401738 wait complete. 2024-03-16T08:08:28.674 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-03-16T08:08:28.674 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:08:28.675 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-03-16T08:08:28.693 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:08:28.693 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:28.749 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-03-16T08:08:28.750 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch host add smithi154 2024-03-16T08:08:29.001 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:29.416 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:29 smithi049 ceph-mon[31942]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:29.416 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:29.416 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:29 smithi049 ceph-mon[31942]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 -- 172.21.15.49:0/1806984847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 msgr2=0x7f12b4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/1806984847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b4104500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009910 tx=0x7f12a4009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 -- 172.21.15.49:0/1806984847 shutdown_connections 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/1806984847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b4104500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 -- 172.21.15.49:0/1806984847 >> 172.21.15.49:0/1806984847 conn(0x7f12b40ff9b0 msgr2=0x7f12b4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 -- 172.21.15.49:0/1806984847 shutdown_connections 2024-03-16T08:08:30.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.477+0000 7f12b9cbe700 1 -- 172.21.15.49:0/1806984847 wait complete. 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b9cbe700 1 Processor -- start 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b9cbe700 1 -- start start 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b9cbe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b9cbe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410e4f0 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51864/0 (socket says 172.21.15.49:51864) 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b37fe700 1 -- 172.21.15.49:0/2688997110 learned_addr learned my addr 172.21.15.49:0/2688997110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.478+0000 7f12b37fe700 1 -- 172.21.15.49:0/2688997110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b419fbf0 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b37fe700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005850 tx=0x7f12a4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12a4026070 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a400bbd0 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b419fee0 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12a4003970 con 0x7f12b4104120 2024-03-16T08:08:30.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.479+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b4199830 con 0x7f12b4104120 2024-03-16T08:08:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f12a4029070 con 0x7f12b4104120 2024-03-16T08:08:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4105600 con 0x7f12b4104120 2024-03-16T08:08:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b0ff9700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 0x7f12a003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f12a401e070 con 0x7f12b4104120 2024-03-16T08:08:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b2ffd700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 0x7f12a003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.480+0000 7f12b2ffd700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 0x7f12a003b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f129c006fd0 tx=0x7f129c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:30.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.485+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f12a4013070 con 0x7f12b4104120 2024-03-16T08:08:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:30.695+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}) -- 0x7f12b419ad60 con 0x7f12a0039020 2024-03-16T08:08:31.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:31 smithi049 ceph-mon[31942]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:32.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:32.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:32.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:32.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:32.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:08:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:08:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:32 smithi049 ceph-mon[31942]: Deploying cephadm binary to smithi154 2024-03-16T08:08:33.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:33 smithi049 ceph-mon[31942]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:33.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:33 smithi049 ceph-mon[31942]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:33.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:33 smithi049 ceph-mon[31942]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:34.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.931+0000 7f12b0ff9700 1 -- 172.21.15.49:0/2688997110 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f12b419ad60 con 0x7f12a0039020 2024-03-16T08:08:34.932 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 msgr2=0x7f12a003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 0x7f12a003b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f129c006fd0 tx=0x7f129c006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 msgr2=0x7f12b419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005850 tx=0x7f12a4005030 comp rx=0 tx=0).stop 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 shutdown_connections 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12a0039020 0x7f12a003b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 --2- 172.21.15.49:0/2688997110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104120 0x7f12b419f6b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.933+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 >> 172.21.15.49:0/2688997110 conn(0x7f12b40ff9b0 msgr2=0x7f12b4105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.934+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 shutdown_connections 2024-03-16T08:08:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:34.934+0000 7f12b9cbe700 1 -- 172.21.15.49:0/2688997110 wait complete. 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:35.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T08:08:35.860 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:36.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:36 smithi049 ceph-mon[31942]: Added host smithi154 2024-03-16T08:08:36.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:36 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:36.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:36 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:37.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 -- 172.21.15.49:0/3194148461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 msgr2=0x7faa401022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:37.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 --2- 172.21.15.49:0/3194148461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa401022f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa30009910 tx=0x7faa30009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:37.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 -- 172.21.15.49:0/3194148461 shutdown_connections 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 --2- 172.21.15.49:0/3194148461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa401022f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 -- 172.21.15.49:0/3194148461 >> 172.21.15.49:0/3194148461 conn(0x7faa400fd7e0 msgr2=0x7faa400ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 -- 172.21.15.49:0/3194148461 shutdown_connections 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 -- 172.21.15.49:0/3194148461 wait complete. 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.289+0000 7faa47092700 1 Processor -- start 2024-03-16T08:08:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa47092700 1 -- start start 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa47092700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa47092700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faa4010c2e0 con 0x7faa40101f10 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa44e2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa44e2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44624/0 (socket says 172.21.15.49:44624) 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa44e2e700 1 -- 172.21.15.49:0/4225886938 learned_addr learned my addr 172.21.15.49:0/4225886938 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa44e2e700 1 -- 172.21.15.49:0/4225886938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4019db20 con 0x7faa40101f10 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa44e2e700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa30005850 tx=0x7faa30005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.290+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30026070 con 0x7faa40101f10 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.291+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3000bbd0 con 0x7faa40101f10 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.291+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4019de10 con 0x7faa40101f10 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.291+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa30003970 con 0x7faa40101f10 2024-03-16T08:08:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.291+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa40197880 con 0x7faa40101f10 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7faa30029070 con 0x7faa40101f10 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa401033f0 con 0x7faa40101f10 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa3dffb700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 0x7faa2803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa3001e070 con 0x7faa40101f10 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa3ffff700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 0x7faa2803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.292+0000 7faa3ffff700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 0x7faa2803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.297+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faa30013070 con 0x7faa40101f10 2024-03-16T08:08:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.508+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa40198be0 con 0x7faa28039020 2024-03-16T08:08:37.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:37 smithi049 ceph-mon[31942]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:37.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.509+0000 7faa3dffb700 1 -- 172.21.15.49:0/4225886938 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7faa40198be0 con 0x7faa28039020 2024-03-16T08:08:37.509 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:08:37.509 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 msgr2=0x7faa2803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 0x7faa2803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 msgr2=0x7faa4019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa30005850 tx=0x7faa30005030 comp rx=0 tx=0).stop 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 shutdown_connections 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faa28039020 0x7faa2803b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 --2- 172.21.15.49:0/4225886938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40101f10 0x7faa4019d5e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 >> 172.21.15.49:0/4225886938 conn(0x7faa400fd7e0 msgr2=0x7faa40104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 shutdown_connections 2024-03-16T08:08:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:37.511+0000 7faa47092700 1 -- 172.21.15.49:0/4225886938 wait complete. 2024-03-16T08:08:38.069 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-16T08:08:38.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-16T08:08:38.316 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:08:39.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:39 smithi049 ceph-mon[31942]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:39.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:39 smithi049 ceph-mon[31942]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:08:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 -- 172.21.15.49:0/1361048056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 msgr2=0x7f7c1c1022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 --2- 172.21.15.49:0/1361048056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c1022f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009910 tx=0x7f7c10009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 -- 172.21.15.49:0/1361048056 shutdown_connections 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 --2- 172.21.15.49:0/1361048056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c1022f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 -- 172.21.15.49:0/1361048056 >> 172.21.15.49:0/1361048056 conn(0x7f7c1c0fd7c0 msgr2=0x7f7c1c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 -- 172.21.15.49:0/1361048056 shutdown_connections 2024-03-16T08:08:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.780+0000 7f7c22d44700 1 -- 172.21.15.49:0/1361048056 wait complete. 2024-03-16T08:08:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c22d44700 1 Processor -- start 2024-03-16T08:08:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c22d44700 1 -- start start 2024-03-16T08:08:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c22d44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c22d44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c10c2e0 con 0x7f7c1c101f10 2024-03-16T08:08:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c20ae0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c20ae0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44640/0 (socket says 172.21.15.49:44640) 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.781+0000 7f7c20ae0700 1 -- 172.21.15.49:0/2788140548 learned_addr learned my addr 172.21.15.49:0/2788140548 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c20ae0700 1 -- 172.21.15.49:0/2788140548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c19d990 con 0x7f7c1c101f10 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c20ae0700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c10005850 tx=0x7f7c10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c10026070 con 0x7f7c1c101f10 2024-03-16T08:08:39.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1000bbd0 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c19dc80 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c10003970 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.782+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c197610 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.783+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7c10029070 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.783+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c1033f0 con 0x7f7c1c101f10 2024-03-16T08:08:39.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.783+0000 7f7c19ffb700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 0x7f7c0c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:39.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.783+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c1001e070 con 0x7f7c1c101f10 2024-03-16T08:08:39.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.783+0000 7f7c1bfff700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 0x7f7c0c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:39.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.784+0000 7f7c1bfff700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 0x7f7c0c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:39.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.788+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7c10013070 con 0x7f7c1c101f10 2024-03-16T08:08:39.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:39.998+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7c1c198b40 con 0x7f7c1c101f10 2024-03-16T08:08:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.273+0000 7f7c19ffb700 1 -- 172.21.15.49:0/2788140548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7c10030020 con 0x7f7c1c101f10 2024-03-16T08:08:40.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 msgr2=0x7f7c0c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:40.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 0x7f7c0c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 msgr2=0x7f7c1c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c10005850 tx=0x7f7c10005030 comp rx=0 tx=0).stop 2024-03-16T08:08:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 shutdown_connections 2024-03-16T08:08:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7c0c038fc0 0x7f7c0c03b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 --2- 172.21.15.49:0/2788140548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101f10 0x7f7c1c19d450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 >> 172.21.15.49:0/2788140548 conn(0x7f7c1c0fd7c0 msgr2=0x7f7c1c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 shutdown_connections 2024-03-16T08:08:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:08:40.277+0000 7f7c22d44700 1 -- 172.21.15.49:0/2788140548 wait complete. 2024-03-16T08:08:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-03-16T08:08:40.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:40 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2788140548' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T08:08:40.913 INFO:tasks.cephadm:Adding mon.a on smithi049 2024-03-16T08:08:40.913 INFO:tasks.cephadm:Adding mon.b on smithi129 2024-03-16T08:08:40.914 INFO:tasks.cephadm:Adding mon.c on smithi154 2024-03-16T08:08:40.914 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c' 2024-03-16T08:08:41.150 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:41.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:41.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2788140548' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- 172.21.15.154:0/503667480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 msgr2=0x7feba81011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 --2- 172.21.15.154:0/503667480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81011d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009910 tx=0x7feb9c009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- 172.21.15.154:0/503667480 shutdown_connections 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 --2- 172.21.15.154:0/503667480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81011d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- 172.21.15.154:0/503667480 >> 172.21.15.154:0/503667480 conn(0x7feba80fc6a0 msgr2=0x7feba80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- 172.21.15.154:0/503667480 shutdown_connections 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- 172.21.15.154:0/503667480 wait complete. 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 Processor -- start 2024-03-16T08:08:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.527+0000 7febaebaa700 1 -- start start 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febaebaa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febaebaa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feba810a0b0 con 0x7feba8100df0 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febac946700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febac946700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.154:35140/0 (socket says 172.21.15.154:35140) 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febac946700 1 -- 172.21.15.154:0/510552404 learned_addr learned my addr 172.21.15.154:0/510552404 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febac946700 1 -- 172.21.15.154:0/510552404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8195af0 con 0x7feba8100df0 2024-03-16T08:08:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.528+0000 7febac946700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005850 tx=0x7feb9c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.529+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb9c026070 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.529+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba8195de0 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.529+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba8199e80 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.529+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c00bbd0 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.529+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb9c0039b0 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.530+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7feb9c029070 con 0x7feba8100df0 2024-03-16T08:08:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.530+0000 7feba5ffb700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 0x7feb9803d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.530+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba81022d0 con 0x7feba8100df0 2024-03-16T08:08:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.530+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feb9c01e070 con 0x7feba8100df0 2024-03-16T08:08:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.530+0000 7feba7fff700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 0x7feb9803d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.531+0000 7feba7fff700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 0x7feb9803d6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:42.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.535+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb9c013070 con 0x7feba8100df0 2024-03-16T08:08:42.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:42 smithi049 ceph-mon[31942]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:42.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:42 smithi049 ceph-mon[31942]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:42.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:42 smithi049 ceph-mon[31942]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:08:42.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.741+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}) -- 0x7feba8196910 con 0x7feb9803b220 2024-03-16T08:08:42.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.747+0000 7feba5ffb700 1 -- 172.21.15.154:0/510552404 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feba8196910 con 0x7feb9803b220 2024-03-16T08:08:42.747 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mon update... 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 msgr2=0x7feb9803d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 0x7feb9803d6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 msgr2=0x7feba81955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005850 tx=0x7feb9c005030 comp rx=0 tx=0).stop 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 shutdown_connections 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9803b220 0x7feb9803d6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 --2- 172.21.15.154:0/510552404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feba8100df0 0x7feba81955b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 >> 172.21.15.154:0/510552404 conn(0x7feba80fc6a0 msgr2=0x7feba81027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 shutdown_connections 2024-03-16T08:08:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:42.750+0000 7febaebaa700 1 -- 172.21.15.154:0/510552404 wait complete. 2024-03-16T08:08:43.297 DEBUG:teuthology.orchestra.run.smithi129:mon.b> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.b.service 2024-03-16T08:08:43.301 DEBUG:teuthology.orchestra.run.smithi154:mon.c> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.c.service 2024-03-16T08:08:43.303 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T08:08:43.303 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:08:43.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:08:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:44 smithi049 ceph-mon[31942]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:44 smithi049 ceph-mon[31942]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c;count:3 2024-03-16T08:08:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:44 smithi049 ceph-mon[31942]: Deploying daemon mon.c on smithi154 2024-03-16T08:08:45.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:45 smithi049 ceph-mon[31942]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:46.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 -- 172.21.15.154:0/737845839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 msgr2=0x7f2d98103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 --2- 172.21.15.154:0/737845839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d98103240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009910 tx=0x7f2d88009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 -- 172.21.15.154:0/737845839 shutdown_connections 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 --2- 172.21.15.154:0/737845839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d98103240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 -- 172.21.15.154:0/737845839 >> 172.21.15.154:0/737845839 conn(0x7f2d980fc690 msgr2=0x7f2d980feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 -- 172.21.15.154:0/737845839 shutdown_connections 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.662+0000 7f2da0461700 1 -- 172.21.15.154:0/737845839 wait complete. 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.663+0000 7f2da0461700 1 Processor -- start 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.663+0000 7f2da0461700 1 -- start start 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.663+0000 7f2da0461700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.663+0000 7f2da0461700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810c330 con 0x7f2d98102e60 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d9e1fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:46.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d9e1fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.154:35158/0 (socket says 172.21.15.154:35158) 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d9e1fd700 1 -- 172.21.15.154:0/755498479 learned_addr learned my addr 172.21.15.154:0/755498479 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d9e1fd700 1 -- 172.21.15.154:0/755498479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9819d9a0 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d9e1fd700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d88005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.664+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d88026070 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.665+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9819dc30 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.665+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8800bc30 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.665+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d88003ab0 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.665+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d98197620 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.666+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9806a990 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.666+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f2d88029070 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.666+0000 7f2d8f7fe700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 0x7f2d8403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.666+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2d8801e070 con 0x7f2d98102e60 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.666+0000 7f2d9d9fc700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 0x7f2d8403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.667+0000 7f2d9d9fc700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 0x7f2d8403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:46.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.671+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2d88050030 con 0x7f2d98102e60 2024-03-16T08:08:46.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.973+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2d98063900 con 0x7f2d98102e60 2024-03-16T08:08:46.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.974+0000 7f2d8f7fe700 1 -- 172.21.15.154:0/755498479 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2d88030020 con 0x7f2d98102e60 2024-03-16T08:08:46.974 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:08:46.974 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":1,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","modified":"2024-03-16T08:07:05.779270Z","created":"2024-03-16T08:07:05.779270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 msgr2=0x7f2d8403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 0x7f2d8403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006e40 comp rx=0 tx=0).stop 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 msgr2=0x7f2d9819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d88005050 comp rx=0 tx=0).stop 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 shutdown_connections 2024-03-16T08:08:46.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d84038fc0 0x7f2d8403b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 --2- 172.21.15.154:0/755498479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98102e60 0x7f2d9819d460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 >> 172.21.15.154:0/755498479 conn(0x7f2d980fc690 msgr2=0x7f2d98103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 shutdown_connections 2024-03-16T08:08:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:46.976+0000 7f2da0461700 1 -- 172.21.15.154:0/755498479 wait complete. 2024-03-16T08:08:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 1 2024-03-16T08:08:47.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:47 smithi049 ceph-mon[31942]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:47.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:47 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/755498479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:08:48.177 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 podman[36452]: 2024-03-16 08:08:48.17774511 +0000 UTC m=+0.360833300 container init 28848fa0265bda66917a5d516db7b7b6c4c718a066aa3f11596df563bcaf5c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-16T08:08:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 podman[36452]: 2024-03-16 08:08:48.180689872 +0000 UTC m=+0.363778053 container start 28848fa0265bda66917a5d516db7b7b6c4c718a066aa3f11596df563bcaf5c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307) 2024-03-16T08:08:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T08:08:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T08:08:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pidfile_write: ignore empty --pid-file 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: load: jerasure load: lrc 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: RocksDB version: 7.9.2 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Git sha 0 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: DB SUMMARY 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: DB Session ID: PVM9B33WTCA7OEJA6ARL 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: CURRENT file: CURRENT 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.error_if_exists: 0 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.create_if_missing: 0 2024-03-16T08:08:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.paranoid_checks: 1 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.env: 0x5603d5526c60 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.info_log: 0x5603d61cbac0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.statistics: (nil) 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.use_fsync: 0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_log_file_size: 0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T08:08:48.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_fallocate: 1 2024-03-16T08:08:48.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T08:08:48.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T08:08:48.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.use_direct_reads: 0 2024-03-16T08:08:48.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.db_log_dir: 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.wal_dir: 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.write_buffer_manager: 0x5603d619e1e0 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T08:08:48.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.unordered_write: 0 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.row_cache: None 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.wal_filter: None 2024-03-16T08:08:48.481 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.two_write_queues: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.wal_compression: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.atomic_flush: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.log_readahead_size: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T08:08:48.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_background_jobs: 2 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_background_compactions: -1 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_subcompactions: 1 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T08:08:48.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_open_files: -1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_background_flushes: -1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Compression algorithms supported: 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kZSTD supported: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kXpressCompression supported: 0 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kZlibCompression supported: 1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kSnappyCompression supported: 1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kLZ4Compression supported: 1 2024-03-16T08:08:48.484 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: kBZip2Compression supported: 0 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.merge_operator: 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_filter: None 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5603d61cbbc0) 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout: cache_index_and_filter_blocks: 1 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi154.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: pin_top_level_index_and_filter: 1 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_type: 0 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: data_block_index_type: 0 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_shortening: 1 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: checksum: 4 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: no_block_cache: 0 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache: 0x5603d61d1090 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_name: BinnedLRUCache 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_options: 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: capacity : 536870912 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: num_shard_bits : 4 2024-03-16T08:08:48.486 INFO:journalctl@ceph.mon.c.smithi154.stdout: strict_capacity_limit : 0 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: high_pri_pool_ratio: 0.000 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_compressed: (nil) 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: persistent_cache: (nil) 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_size: 4096 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_size_deviation: 10 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_restart_interval: 16 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_block_restart_interval: 1 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: metadata_block_size: 4096 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: partition_filters: 0 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: use_delta_encoding: 1 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: filter_policy: bloomfilter 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: whole_key_filtering: 1 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: verify_compression: 0 2024-03-16T08:08:48.487 INFO:journalctl@ceph.mon.c.smithi154.stdout: read_amp_bytes_per_bit: 0 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: format_version: 5 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: enable_index_compression: 1 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_align: 0 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: max_auto_readahead_size: 262144 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: prepopulate_block_cache: 0 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: initial_auto_readahead_size: 8192 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression: NoCompression 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.num_levels: 7 2024-03-16T08:08:48.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T08:08:48.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T08:08:48.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T08:08:48.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.table_properties_collectors: 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.inplace_update_support: 0 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T08:08:48.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.bloom_locality: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.max_successive_merges: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.ttl: 2592000 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enable_blob_files: false 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.min_blob_size: 0 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T08:08:48.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8489017e-8205-41be-9299-27775a7a144a 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576528210341, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576528210769, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710576528, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8489017e-8205-41be-9299-27775a7a144a", "db_session_id": "PVM9B33WTCA7OEJA6ARL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-16T08:08:48.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710576528210847, "job": 1, "event": "recovery_finished"} 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5603d62c4000 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: DB pointer 0x5603d62ae000 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** DB Stats ** 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:08:48.495 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** Compaction Stats [default] ** 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** Compaction Stats [default] ** 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T08:08:48.496 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Block cache BinnedLRUCache@0x5603d61d1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: using public_addr v2:172.21.15.154:0/0 -> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-03-16T08:08:48.497 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: starting mon.c rank -1 at public addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] at bind addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(???) e0 preinit fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).mds e1 new map 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: e1 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: legacy client fscid: -1 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout: No filesystems configured 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-16T08:08:48.498 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.499 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Deploying cephadm binary to smithi154 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.500 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Added host smithi154 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2788140548' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2788140548' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.501 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.502 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c;count:3 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: Deploying daemon mon.c on smithi154 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/755498479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 ceph-mon[36471]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 bash[36452]: 28848fa0265bda66917a5d516db7b7b6c4c718a066aa3f11596df563bcaf5c3f 2024-03-16T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:48 smithi154 systemd[1]: Started Ceph mon.c for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:08:48.741 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T08:08:48.742 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:08:48.975 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:08:53.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 -- 172.21.15.154:0/45609384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74102f60 msgr2=0x7f9f60005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 --2- 172.21.15.154:0/45609384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74102f60 0x7f9f60005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005fa0 tx=0x7f9f6400ffc0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 -- 172.21.15.154:0/45609384 shutdown_connections 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 --2- 172.21.15.154:0/45609384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74102f60 0x7f9f60005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 -- 172.21.15.154:0/45609384 >> 172.21.15.154:0/45609384 conn(0x7f9f740fc790 msgr2=0x7f9f740febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 -- 172.21.15.154:0/45609384 shutdown_connections 2024-03-16T08:08:53.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.320+0000 7f9f793d0700 1 -- 172.21.15.154:0/45609384 wait complete. 2024-03-16T08:08:53.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.321+0000 7f9f793d0700 1 Processor -- start 2024-03-16T08:08:53.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.321+0000 7f9f793d0700 1 -- start start 2024-03-16T08:08:53.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f793d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f793d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 0x7f9f74101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f793d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74109e40 con 0x7f9f74101100 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f793d0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74109cc0 con 0x7f9f74102f60 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57052/0 (socket says 172.21.15.154:57052) 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 -- 172.21.15.154:0/3728855762 learned_addr learned my addr 172.21.15.154:0/3728855762 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f6a7fc700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 0x7f9f74101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 -- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 msgr2=0x7f9f74100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T08:08:53.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 -- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 msgr2=0x7f9f74100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f72ffd700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f6a7fc700 1 -- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 msgr2=0x7f9f74100bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f6a7fc700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.322+0000 7f9f6a7fc700 1 -- 172.21.15.154:0/3728855762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74102ae0 con 0x7f9f74101100 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f6a7fc700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 0x7f9f74101580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00b4d0 tx=0x7f9f5c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9f5c011630 con 0x7f9f74101100 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c011cb0 con 0x7f9f74101100 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9f5c010de0 con 0x7f9f74101100 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f741a3c80 con 0x7f9f74101100 2024-03-16T08:08:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.323+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f741a41c0 con 0x7f9f74101100 2024-03-16T08:08:53.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.324+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f9f5c018020 con 0x7f9f74101100 2024-03-16T08:08:53.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.325+0000 7f9f70ff9700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 0x7f9f5403b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:53.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.325+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f5c04b2d0 con 0x7f9f74101100 2024-03-16T08:08:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.325+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7406ad90 con 0x7f9f74101100 2024-03-16T08:08:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.325+0000 7f9f72ffd700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 0x7f9f5403b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.325+0000 7f9f72ffd700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 0x7f9f5403b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005fd0 tx=0x7f9f6401d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.330+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f5c01f070 con 0x7f9f74101100 2024-03-16T08:08:53.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.610+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9f741a44a0 con 0x7f9f74101100 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: mon.a calling monitor election 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: mon.c calling monitor election 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: monmap epoch 2 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: last_changed 2024-03-16T08:08:48.233627+0000 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: min_mon_release 19 (squid) 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: election_strategy: 1 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: fsmap 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: mgrmap e13: x(active, since 49s) 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: overall HEALTH_OK 2024-03-16T08:08:53.612 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:53.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:53.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.611+0000 7f9f70ff9700 1 -- 172.21.15.154:0/3728855762 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f9f5c00e260 con 0x7f9f74101100 2024-03-16T08:08:53.613 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:08:53.613 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","modified":"2024-03-16T08:08:48.233627Z","created":"2024-03-16T08:07:05.779270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 msgr2=0x7f9f5403b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 0x7f9f5403b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005fd0 tx=0x7f9f6401d040 comp rx=0 tx=0).stop 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 msgr2=0x7f9f74101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 0x7f9f74101580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00b4d0 tx=0x7f9f5c00b890 comp rx=0 tx=0).stop 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 shutdown_connections 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9f54038d80 0x7f9f5403b240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f74102f60 0x7f9f74100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 --2- 172.21.15.154:0/3728855762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f74101100 0x7f9f74101580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 >> 172.21.15.154:0/3728855762 conn(0x7f9f740fc790 msgr2=0x7f9f74103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:53.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 shutdown_connections 2024-03-16T08:08:53.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:53.613+0000 7f9f793d0700 1 -- 172.21.15.154:0/3728855762 wait complete. 2024-03-16T08:08:53.615 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 2 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: mon.a calling monitor election 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: mon.c calling monitor election 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T08:08:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: monmap epoch 2 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: last_changed 2024-03-16T08:08:48.233627+0000 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: min_mon_release 19 (squid) 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: election_strategy: 1 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: fsmap 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: mgrmap e13: x(active, since 49s) 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: overall HEALTH_OK 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:53.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:54.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:08:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: Deploying daemon mon.b on smithi129 2024-03-16T08:08:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/3728855762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:08:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:54.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:08:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:08:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:08:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: Deploying daemon mon.b on smithi129 2024-03-16T08:08:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/3728855762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:08:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:08:55.231 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T08:08:55.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:08:55.465 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:08:55.586 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:08:55 smithi154 ceph-mon[36471]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:55.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:08:55 smithi049 ceph-mon[31942]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:08:56.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 -- 172.21.15.154:0/896750372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370100f40 msgr2=0x7f8370101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:08:56.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 --2- 172.21.15.154:0/896750372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370100f40 0x7f8370101320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8360009910 tx=0x7f8360009c20 comp rx=0 tx=0).stop 2024-03-16T08:08:56.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 -- 172.21.15.154:0/896750372 shutdown_connections 2024-03-16T08:08:56.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 --2- 172.21.15.154:0/896750372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370100f40 0x7f8370101320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:56.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 -- 172.21.15.154:0/896750372 >> 172.21.15.154:0/896750372 conn(0x7f83700fc7f0 msgr2=0x7f83700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:08:56.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 -- 172.21.15.154:0/896750372 shutdown_connections 2024-03-16T08:08:56.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.981+0000 7f83756ac700 1 -- 172.21.15.154:0/896750372 wait complete. 2024-03-16T08:08:56.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 Processor -- start 2024-03-16T08:08:56.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 -- start start 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f837010b310 con 0x7f837019dc60 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f83756ac700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f837010b190 con 0x7f8370100f40 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f836e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f836effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.982+0000 7f836e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.154:45334/0 (socket says 172.21.15.154:45334) 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836e7fc700 1 -- 172.21.15.154:0/2544985864 learned_addr learned my addr 172.21.15.154:0/2544985864 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836effd700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57056/0 (socket says 172.21.15.154:57056) 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836effd700 1 -- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 msgr2=0x7f837019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836effd700 1 -- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 msgr2=0x7f837019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836effd700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836effd700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:08:56.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836e7fc700 1 -- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 msgr2=0x7f837019d720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836e7fc700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836e7fc700 1 -- 172.21.15.154:0/2544985864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837019e2a0 con 0x7f837019dc60 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f836e7fc700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f836400b700 tx=0x7f836400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8364011630 con 0x7f837019dc60 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8370197e20 con 0x7f837019dc60 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8364011cb0 con 0x7f837019dc60 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8364010e60 con 0x7f837019dc60 2024-03-16T08:08:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.983+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701983b0 con 0x7f837019dc60 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.984+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8370102420 con 0x7f837019dc60 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.984+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8364018020 con 0x7f837019dc60 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.985+0000 7f8357fff700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 0x7f835803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.985+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f836404bb60 con 0x7f837019dc60 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.985+0000 7f836effd700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 0x7f835803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:08:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.985+0000 7f836effd700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 0x7f835803b1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f836000b5c0 tx=0x7f8360005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:08:56.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:56.990+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f836401f020 con 0x7f837019dc60 2024-03-16T08:08:57.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:57.202+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364010510 con 0x7f837019dc60 2024-03-16T08:08:57.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:08:57.268+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8370198eb0 con 0x7f837019dc60 2024-03-16T08:09:02.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.234+0000 7f8357fff700 1 -- 172.21.15.154:0/2544985864 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f836401a920 con 0x7f837019dc60 2024-03-16T08:09:02.234 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-16T08:09:02.234 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","modified":"2024-03-16T08:08:57.202322Z","created":"2024-03-16T08:07:05.779270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 msgr2=0x7f835803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 0x7f835803b1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f836000b5c0 tx=0x7f8360005a90 comp rx=0 tx=0).stop 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 msgr2=0x7f83701978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f836400b700 tx=0x7f836400bac0 comp rx=0 tx=0).stop 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 shutdown_connections 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8358038d00 0x7f835803b1c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8370100f40 0x7f837019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 --2- 172.21.15.154:0/2544985864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f837019dc60 0x7f83701978e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:02.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 >> 172.21.15.154:0/2544985864 conn(0x7f83700fc7f0 msgr2=0x7f837006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:02.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 shutdown_connections 2024-03-16T08:09:02.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:02.237+0000 7f83756ac700 1 -- 172.21.15.154:0/2544985864 wait complete. 2024-03-16T08:09:02.238 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 3 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: mon.a calling monitor election 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: mon.c calling monitor election 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:09:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/2544985864' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: monmap epoch 3 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: last_changed 2024-03-16T08:08:57.202322+0000 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:09:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: min_mon_release 19 (squid) 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: election_strategy: 1 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: fsmap 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: mgrmap e13: x(active, since 58s) 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: mon.b (rank 2) addr [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] is down (out of quorum) 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:09:02.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: mon.a calling monitor election 2024-03-16T08:09:02.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: mon.c calling monitor election 2024-03-16T08:09:02.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/2544985864' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: monmap epoch 3 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:02.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: last_changed 2024-03-16T08:08:57.202322+0000 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: min_mon_release 19 (squid) 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: election_strategy: 1 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: fsmap 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: mgrmap e13: x(active, since 58s) 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: mon.b (rank 2) addr [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] is down (out of quorum) 2024-03-16T08:09:02.570 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.571 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.571 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:02.836 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-16T08:09:02.837 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-16T08:09:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:09:03.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:03 smithi049 ceph-mon[31942]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:03.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:03.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:03.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:03.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:03 smithi154 ceph-mon[36471]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:03.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:03.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:04.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 -- 172.21.15.49:0/1158176801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe504100e00 msgr2=0x7fe5041011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 --2- 172.21.15.49:0/1158176801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe504100e00 0x7fe5041011e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009910 tx=0x7fe4f4009c20 comp rx=0 tx=0).stop 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 -- 172.21.15.49:0/1158176801 shutdown_connections 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 --2- 172.21.15.49:0/1158176801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe504100e00 0x7fe5041011e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 -- 172.21.15.49:0/1158176801 >> 172.21.15.49:0/1158176801 conn(0x7fe504077810 msgr2=0x7fe5040feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 -- 172.21.15.49:0/1158176801 shutdown_connections 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.770+0000 7fe509a7e700 1 -- 172.21.15.49:0/1158176801 wait complete. 2024-03-16T08:09:04.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 Processor -- start 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 -- start start 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe504100e00 0x7fe5041028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe504102de0 0x7fe504103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410b190 con 0x7fe5041a3d70 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410b010 con 0x7fe504102de0 2024-03-16T08:09:04.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe509a7e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410b310 con 0x7fe504100e00 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe503fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe503fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60324/0 (socket says 172.21.15.49:60324) 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.771+0000 7fe503fff700 1 -- 172.21.15.49:0/2887905843 learned_addr learned my addr 172.21.15.49:0/2887905843 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe5037fe700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe504100e00 0x7fe5041028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe502ffd700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe504102de0 0x7fe504103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 -- 172.21.15.49:0/2887905843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe504100e00 msgr2=0x7fe5041028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe504100e00 0x7fe5041028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 -- 172.21.15.49:0/2887905843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe504102de0 msgr2=0x7fe504103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe504102de0 0x7fe504103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 -- 172.21.15.49:0/2887905843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041a6830 con 0x7fe5041a3d70 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe503fff700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ba50 tx=0x7fe4fc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:04.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc019070 con 0x7fe5041a3d70 2024-03-16T08:09:04.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc00ce10 con 0x7fe5041a3d70 2024-03-16T08:09:04.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041a6b20 con 0x7fe5041a3d70 2024-03-16T08:09:04.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc007370 con 0x7fe5041a3d70 2024-03-16T08:09:04.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.772+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a7010 con 0x7fe5041a3d70 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.773+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe4fc01c070 con 0x7fe5041a3d70 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.773+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe504063900 con 0x7fe5041a3d70 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.774+0000 7fe500ff9700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 0x7fe4ec03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.774+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4fc04b540 con 0x7fe5041a3d70 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.774+0000 7fe5037fe700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 0x7fe4ec03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:04.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.774+0000 7fe5037fe700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 0x7fe4ec03b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009750 tx=0x7fe4f400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:04.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.779+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe4fc00f160 con 0x7fe5041a3d70 2024-03-16T08:09:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.995+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe5041a7270 con 0x7fe5041a3d70 2024-03-16T08:09:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.995+0000 7fe500ff9700 1 -- 172.21.15.49:0/2887905843 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fe4fc014440 con 0x7fe5041a3d70 2024-03-16T08:09:04.996 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:04.996 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-03-16T08:09:04.996 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:04.996 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 msgr2=0x7fe4ec03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 0x7fe4ec03b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009750 tx=0x7fe4f400b540 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 msgr2=0x7fe5041a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ba50 tx=0x7fe4fc00bd60 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 shutdown_connections 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe4ec039450 0x7fe4ec03b910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041a3d70 0x7fe5041a6160 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe504100e00 0x7fe5041028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 --2- 172.21.15.49:0/2887905843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe504102de0 0x7fe504103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.997+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 >> 172.21.15.49:0/2887905843 conn(0x7fe504077810 msgr2=0x7fe504069760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.998+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 shutdown_connections 2024-03-16T08:09:04.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:04.998+0000 7fe509a7e700 1 -- 172.21.15.49:0/2887905843 wait complete. 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mon.b calling monitor election 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mon.b calling monitor election 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mon.c calling monitor election 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mon.a calling monitor election 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: monmap epoch 3 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: last_changed 2024-03-16T08:08:57.202322+0000 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: min_mon_release 19 (squid) 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: election_strategy: 1 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: fsmap 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:09:05.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: mgrmap e13: x(active, since 60s) 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: overall HEALTH_OK 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:05.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:05 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2887905843' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mon.b calling monitor election 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mon.b calling monitor election 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mon.c calling monitor election 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mon.a calling monitor election 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: monmap epoch 3 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: last_changed 2024-03-16T08:08:57.202322+0000 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: min_mon_release 19 (squid) 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: election_strategy: 1 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:09:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: fsmap 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: mgrmap e13: x(active, since 60s) 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: overall HEALTH_OK 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:05.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:05.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:05.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:05 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2887905843' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mon.b calling monitor election 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mon.b calling monitor election 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mon.c calling monitor election 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mon.a calling monitor election 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: monmap epoch 3 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: last_changed 2024-03-16T08:08:57.202322+0000 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: created 2024-03-16T08:07:05.779270+0000 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: min_mon_release 19 (squid) 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: election_strategy: 1 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-03-16T08:09:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: fsmap 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: mgrmap e13: x(active, since 60s) 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: overall HEALTH_OK 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:05.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:05.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:05.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:05.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:05 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2887905843' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:06.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T08:09:06.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: Reconfiguring daemon mon.a on smithi049 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T08:09:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: Reconfiguring daemon mon.a on smithi049 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:06.477 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-16T08:09:06.477 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:09:06.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T08:09:06.513 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:09:06.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:09:06.588 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:09:06.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T08:09:06.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T08:09:06.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: Reconfiguring daemon mon.a on smithi049 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:06.619 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:09:06.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:09:06.685 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:09:06.685 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T08:09:06.720 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:09:06.720 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:09:06.789 INFO:tasks.cephadm:Adding mgr.x on smithi049 2024-03-16T08:09:06.789 INFO:tasks.cephadm:Adding mgr.y on smithi129 2024-03-16T08:09:06.790 INFO:tasks.cephadm:Adding mgr.z on smithi154 2024-03-16T08:09:06.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi049=x;smithi129=y;smithi154=z' 2024-03-16T08:09:07.061 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:09:07.470 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:07 smithi129 ceph-mon[36561]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:07.470 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:07 smithi129 ceph-mon[36561]: Reconfiguring mon.b (monmap changed)... 2024-03-16T08:09:07.470 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:07 smithi129 ceph-mon[36561]: Reconfiguring daemon mon.b on smithi129 2024-03-16T08:09:07.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:07 smithi154 ceph-mon[36471]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:07.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:07 smithi154 ceph-mon[36471]: Reconfiguring mon.b (monmap changed)... 2024-03-16T08:09:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:07 smithi154 ceph-mon[36471]: Reconfiguring daemon mon.b on smithi129 2024-03-16T08:09:07.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:07 smithi049 ceph-mon[31942]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:07 smithi049 ceph-mon[31942]: Reconfiguring mon.b (monmap changed)... 2024-03-16T08:09:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:07 smithi049 ceph-mon[31942]: Reconfiguring daemon mon.b on smithi129 2024-03-16T08:09:08.523 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:08.523 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.524 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.524 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: Reconfiguring mon.c (monmap changed)... 2024-03-16T08:09:08.524 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:08.524 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:08.524 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:08.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: Reconfiguring mon.c (monmap changed)... 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:08 smithi154 ceph-mon[36471]: Reconfiguring daemon mon.c on smithi154 2024-03-16T08:09:08.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:08.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: Reconfiguring mon.c (monmap changed)... 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:08 smithi049 ceph-mon[31942]: Reconfiguring daemon mon.c on smithi154 2024-03-16T08:09:08.859 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:08 smithi129 ceph-mon[36561]: Reconfiguring daemon mon.c on smithi154 2024-03-16T08:09:09.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- 172.21.15.154:0/209396257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 msgr2=0x7fcaa8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:09.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/209396257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa8110ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b0a0 tx=0x7fcaa000b3b0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- 172.21.15.154:0/209396257 shutdown_connections 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/209396257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa8110ca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/209396257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8076300 0x7fcaa8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/209396257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8074d30 0x7fcaa80751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- 172.21.15.154:0/209396257 >> 172.21.15.154:0/209396257 conn(0x7fcaa80fd8b0 msgr2=0x7fcaa80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- 172.21.15.154:0/209396257 shutdown_connections 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- 172.21.15.154:0/209396257 wait complete. 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 Processor -- start 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.573+0000 7fcaaffb2700 1 -- start start 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8074d30 0x7fcaa819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8076300 0x7fcaa819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81144d0 con 0x7fcaa8074d30 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8114350 con 0x7fcaa8076300 2024-03-16T08:09:09.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8114650 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50320/0 (socket says 172.21.15.154:50320) 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 -- 172.21.15.154:0/3705930437 learned_addr learned my addr 172.21.15.154:0/3705930437 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaadd4e700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8074d30 0x7fcaa819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaad54d700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8076300 0x7fcaa819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 -- 172.21.15.154:0/3705930437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8076300 msgr2=0x7fcaa819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8076300 0x7fcaa819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 -- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8074d30 msgr2=0x7fcaa819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8074d30 0x7fcaa819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 -- 172.21.15.154:0/3705930437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa81a66c0 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaae54f700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0004d00 tx=0x7fcaa0004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0021070 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa81a6950 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa0007b80 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00053e0 con 0x7fcaa810e8b0 2024-03-16T08:09:09.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.574+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a6e00 con 0x7fcaa810e8b0 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.575+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fcaa0024070 con 0x7fcaa810e8b0 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.575+0000 7fca9effd700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 0x7fca9403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.575+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa8068f90 con 0x7fcaa810e8b0 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.575+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcaa000f020 con 0x7fcaa810e8b0 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.576+0000 7fcaadd4e700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 0x7fca9403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:09.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.576+0000 7fcaadd4e700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 0x7fca9403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80766e0 tx=0x7fcaa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:09.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.581+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcaa002b020 con 0x7fcaa810e8b0 2024-03-16T08:09:09.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.788+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi129=y;smithi154=z", "target": ["mon-mgr", ""]}) -- 0x7fcaa8062860 con 0x7fca94039200 2024-03-16T08:09:09.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.794+0000 7fca9effd700 1 -- 172.21.15.154:0/3705930437 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcaa8062860 con 0x7fca94039200 2024-03-16T08:09:09.794 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mgr update... 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 msgr2=0x7fca9403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 0x7fca9403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80766e0 tx=0x7fcaa4009260 comp rx=0 tx=0).stop 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 msgr2=0x7fcaa81a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0004d00 tx=0x7fcaa0004d30 comp rx=0 tx=0).stop 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 shutdown_connections 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fca94039200 0x7fca9403b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaa810e8b0 0x7fcaa81a6020 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa8074d30 0x7fcaa819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 --2- 172.21.15.154:0/3705930437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcaa8076300 0x7fcaa819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:09.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 >> 172.21.15.154:0/3705930437 conn(0x7fcaa80fd8b0 msgr2=0x7fcaa8101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:09.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 shutdown_connections 2024-03-16T08:09:09.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:09:09.796+0000 7fcaaffb2700 1 -- 172.21.15.154:0/3705930437 wait complete. 2024-03-16T08:09:10.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:10.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:10.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.375 DEBUG:teuthology.orchestra.run.smithi129:mgr.y> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.y.service 2024-03-16T08:09:10.378 DEBUG:teuthology.orchestra.run.smithi154:mgr.z> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.z.service 2024-03-16T08:09:10.380 INFO:tasks.cephadm:Deploying OSDs... 2024-03-16T08:09:10.380 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:09:10.380 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T08:09:10.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T08:09:10.397 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-03-16T08:09:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:10.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-16 08:08:15.913046750 +0000 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-16 08:02:40.167200321 +0000 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-16 08:02:40.167200321 +0000 2024-03-16T08:09:10.417 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-16 08:02:40.167200321 +0000 2024-03-16T08:09:10.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T08:09:10.487 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-16T08:09:10.487 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-16T08:09:10.487 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.5362e-05 s, 5.4 MB/s 2024-03-16T08:09:10.488 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T08:09:10.545 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-03-16T08:09:10.600 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-16 08:08:15.948045833 +0000 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-16 08:02:40.510191308 +0000 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-16 08:02:40.510191308 +0000 2024-03-16T08:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-16 08:02:40.510191308 +0000 2024-03-16T08:09:10.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T08:09:10.671 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-16T08:09:10.671 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-16T08:09:10.671 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 8.5413e-05 s, 6.0 MB/s 2024-03-16T08:09:10.672 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T08:09:10.727 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-16 08:08:15.980044995 +0000 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-16 08:02:40.860182112 +0000 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-16 08:02:40.860182112 +0000 2024-03-16T08:09:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-16 08:02:40.860182112 +0000 2024-03-16T08:09:10.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T08:09:10.854 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-16T08:09:10.854 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-16T08:09:10.854 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 8.7488e-05 s, 5.9 MB/s 2024-03-16T08:09:10.855 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T08:09:10.911 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-16 08:08:16.012044157 +0000 2024-03-16T08:09:10.966 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-16 08:02:41.203173099 +0000 2024-03-16T08:09:10.967 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-16 08:02:41.203173099 +0000 2024-03-16T08:09:10.967 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-16 08:02:41.203173099 +0000 2024-03-16T08:09:10.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T08:09:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-16T08:09:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-16T08:09:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000103608 s, 4.9 MB/s 2024-03-16T08:09:11.036 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T08:09:11.093 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-03-16T08:09:11.093 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T08:09:11.112 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T08:09:11.112 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-03-16T08:09:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T08:09:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:11.168 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-16T08:09:11.168 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:11.169 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:11.169 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-03-16 08:08:56.661431338 +0000 2024-03-16T08:09:11.169 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-03-16 08:03:29.604931799 +0000 2024-03-16T08:09:11.169 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-03-16 08:03:29.604931799 +0000 2024-03-16T08:09:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-03-16 08:03:29.604931799 +0000 2024-03-16T08:09:11.169 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T08:09:11.237 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-03-16T08:09:11.237 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-03-16T08:09:11.237 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000285668 s, 1.8 MB/s 2024-03-16T08:09:11.238 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T08:09:11.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:11 smithi049 ceph-mon[31942]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:11.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:11 smithi049 ceph-mon[31942]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi129=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:11 smithi049 ceph-mon[31942]: Saving service mgr spec with placement smithi049=x;smithi129=y;smithi154=z;count:3 2024-03-16T08:09:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:11 smithi049 ceph-mon[31942]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T08:09:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:11 smithi049 ceph-mon[31942]: Reconfiguring daemon mgr.x on smithi049 2024-03-16T08:09:11.295 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-03-16T08:09:11.351 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T08:09:11.351 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:11.351 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-16T08:09:11.351 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:11.351 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:11.352 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-03-16 08:08:56.661431338 +0000 2024-03-16T08:09:11.352 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-03-16 08:03:29.952924085 +0000 2024-03-16T08:09:11.352 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-03-16 08:03:29.952924085 +0000 2024-03-16T08:09:11.352 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-03-16 08:03:29.952924085 +0000 2024-03-16T08:09:11.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T08:09:11.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:11 smithi129 ceph-mon[36561]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:11.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:11 smithi129 ceph-mon[36561]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi129=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:11.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:11 smithi129 ceph-mon[36561]: Saving service mgr spec with placement smithi049=x;smithi129=y;smithi154=z;count:3 2024-03-16T08:09:11.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:11 smithi129 ceph-mon[36561]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T08:09:11.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:11 smithi129 ceph-mon[36561]: Reconfiguring daemon mgr.x on smithi049 2024-03-16T08:09:11.386 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-03-16T08:09:11.386 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-03-16T08:09:11.386 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000227155 s, 2.3 MB/s 2024-03-16T08:09:11.387 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T08:09:11.444 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-03-16T08:09:11.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:11 smithi154 ceph-mon[36471]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:11.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:11 smithi154 ceph-mon[36471]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi129=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:11.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:11 smithi154 ceph-mon[36471]: Saving service mgr spec with placement smithi049=x;smithi129=y;smithi154=z;count:3 2024-03-16T08:09:11.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:11 smithi154 ceph-mon[36471]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T08:09:11.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:11 smithi154 ceph-mon[36471]: Reconfiguring daemon mgr.x on smithi049 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-03-16 08:08:56.662431317 +0000 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-03-16 08:03:30.298916415 +0000 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-03-16 08:03:30.298916415 +0000 2024-03-16T08:09:11.499 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-03-16 08:03:30.298916415 +0000 2024-03-16T08:09:11.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T08:09:11.565 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-03-16T08:09:11.565 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-03-16T08:09:11.565 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000245003 s, 2.1 MB/s 2024-03-16T08:09:11.566 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T08:09:11.622 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-03-16 08:08:56.662431317 +0000 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-03-16 08:03:30.630909056 +0000 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-03-16 08:03:30.630909056 +0000 2024-03-16T08:09:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-03-16 08:03:30.630909056 +0000 2024-03-16T08:09:11.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T08:09:11.743 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-03-16T08:09:11.745 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-03-16T08:09:11.745 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000255389 s, 2.0 MB/s 2024-03-16T08:09:11.746 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T08:09:11.802 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-16T08:09:11.802 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T08:09:11.817 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T08:09:11.817 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-16 08:08:47.460699024 +0000 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-16 08:02:47.812448124 +0000 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-16 08:02:47.812448124 +0000 2024-03-16T08:09:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-16 08:02:47.812448124 +0000 2024-03-16T08:09:11.874 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T08:09:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-16T08:09:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-16T08:09:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000267476 s, 1.9 MB/s 2024-03-16T08:09:11.941 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T08:09:11.997 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-16 08:08:47.460699024 +0000 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-16 08:02:48.152438334 +0000 2024-03-16T08:09:12.055 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-16 08:02:48.152438334 +0000 2024-03-16T08:09:12.056 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-16 08:02:48.152438334 +0000 2024-03-16T08:09:12.056 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T08:09:12.121 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-16T08:09:12.121 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-16T08:09:12.121 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000195197 s, 2.6 MB/s 2024-03-16T08:09:12.122 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T08:09:12.178 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-16 08:08:47.461698998 +0000 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-16 08:02:48.487428564 +0000 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-16 08:02:48.487428564 +0000 2024-03-16T08:09:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-16 08:02:48.487428564 +0000 2024-03-16T08:09:12.236 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T08:09:12.303 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-16T08:09:12.303 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-16T08:09:12.303 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000307731 s, 1.7 MB/s 2024-03-16T08:09:12.305 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T08:09:12.360 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:09:12.415 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-16 08:08:47.461698998 +0000 2024-03-16T08:09:12.416 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-16 08:02:48.834418443 +0000 2024-03-16T08:09:12.416 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-16 08:02:48.834418443 +0000 2024-03-16T08:09:12.416 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-16 08:02:48.834418443 +0000 2024-03-16T08:09:12.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T08:09:12.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: Deploying daemon mgr.z on smithi154 2024-03-16T08:09:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:12 smithi154 ceph-mon[36471]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:12.482 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-16T08:09:12.482 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-16T08:09:12.482 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000211997 s, 2.4 MB/s 2024-03-16T08:09:12.483 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T08:09:12.539 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-03-16T08:09:12.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T08:09:12.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: Deploying daemon mgr.z on smithi154 2024-03-16T08:09:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:12 smithi049 ceph-mon[31942]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:09:12.859 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: Deploying daemon mgr.z on smithi154 2024-03-16T08:09:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:12 smithi129 ceph-mon[36561]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:13.469 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 systemd[1]: Starting Ceph mgr.z for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:09:13.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 podman[38348]: 2024-03-16 08:09:13.516772944 +0000 UTC m=+0.017532565 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:09:13.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 podman[38348]: 2024-03-16 08:09:13.625919022 +0000 UTC m=+0.126678601 container create 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-16T08:09:14.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 podman[38348]: 2024-03-16 08:09:13.853579056 +0000 UTC m=+0.354338655 container init 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T08:09:14.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 podman[38348]: 2024-03-16 08:09:13.856056983 +0000 UTC m=+0.356816569 container start 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T08:09:14.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 ceph-mgr[38368]: -- 172.21.15.154:0/2288279128 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b021d525a0 con 0x55b021d26800 2024-03-16T08:09:14.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 bash[38348]: 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 2024-03-16T08:09:14.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:13 smithi154 systemd[1]: Started Ceph mgr.z for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:09:14.488 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:14.395+0000 7f81e9058200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:09:14.488 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:14.488+0000 7f81e9058200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00831116 s, 1.3 GB/s 2024-03-16T08:09:14.750 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-16T08:09:15.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:15.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:15.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:15 smithi049 ceph-mon[31942]: Deploying daemon mgr.y on smithi129 2024-03-16T08:09:15.359 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.125+0000 7f81e9058200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:09:15.359 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.215+0000 7f81e9058200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:09:15.359 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.358+0000 7f81e9058200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:09:15.359 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-mon[36471]: Deploying daemon mgr.y on smithi129 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:15 smithi129 ceph-mon[36561]: Deploying daemon mgr.y on smithi129 2024-03-16T08:09:15.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-03-16T08:09:15.629 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:09:15.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.454+0000 7f81e9058200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:09:15.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.536+0000 7f81e9058200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:09:16.084 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.797+0000 7f81e9058200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:09:16.084 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.905+0000 7f81e9058200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:09:16.084 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:15.989+0000 7f81e9058200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:09:16.084 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:16.084+0000 7f81e9058200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:09:16.109 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:15 smithi129 systemd[1]: Starting Ceph mgr.y for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 podman[37966]: 2024-03-16 08:09:16.060948984 +0000 UTC m=+0.018098619 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 podman[37966]: 2024-03-16 08:09:16.170485763 +0000 UTC m=+0.127635392 container create 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 podman[37966]: 2024-03-16 08:09:16.372897169 +0000 UTC m=+0.330046799 container init 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 podman[37966]: 2024-03-16 08:09:16.376603804 +0000 UTC m=+0.333753432 container start 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 ceph-mgr[37986]: -- 172.21.15.129:0/1376505992 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56536b7925a0 con 0x56536b766800 2024-03-16T08:09:16.447 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 bash[37966]: 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 2024-03-16T08:09:16.474 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:16.259+0000 7f81e9058200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:09:16.714 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 systemd[1]: Started Ceph mgr.y for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:09:17.007 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:16.955+0000 7f75a4981200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3180470995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628102e00 msgr2=0x7fe6281096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3180470995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628102e00 0x7fe6281096b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe62000b0d0 tx=0x7fe62000b3e0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3180470995 shutdown_connections 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3180470995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628102e00 0x7fe6281096b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3180470995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628101260 0x7fe628101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3180470995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 0x7fe6281028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3180470995 >> 172.21.15.49:0/3180470995 conn(0x7fe6280fc820 msgr2=0x7fe6280fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3180470995 shutdown_connections 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.103+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3180470995 wait complete. 2024-03-16T08:09:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 Processor -- start 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 -- start start 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 0x7fe628103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 0x7fe628103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810cdd0 con 0x7fe628101260 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810cc50 con 0x7fe628102460 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62f6ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810cf50 con 0x7fe628102e00 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62d48a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62d48a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52284/0 (socket says 172.21.15.49:52284) 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62d48a700 1 -- 172.21.15.49:0/3176299347 learned_addr learned my addr 172.21.15.49:0/3176299347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62dc8b700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 0x7fe628103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.104+0000 7fe62cc89700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 0x7fe628103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 -- 172.21.15.49:0/3176299347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 msgr2=0x7fe628103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 0x7fe628103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 -- 172.21.15.49:0/3176299347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 msgr2=0x7fe628103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 0x7fe628103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 -- 172.21.15.49:0/3176299347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe628104630 con 0x7fe628101260 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62dc8b700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 0x7fe628103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62d48a700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d980 tx=0x7fe61c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c017030 con 0x7fe628101260 2024-03-16T08:09:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6281a2610 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c004500 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0039e0 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.105+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6281a2b50 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.106+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe61c01a070 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.106+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe628068f90 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.106+0000 7fe61a7fc700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 0x7fe61403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.106+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe61c04b9d0 con 0x7fe628101260 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.106+0000 7fe62cc89700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 0x7fe61403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.107+0000 7fe62cc89700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 0x7fe61403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe624005950 tx=0x7fe62400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.111+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe61c0092c0 con 0x7fe628101260 2024-03-16T08:09:17.257 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:17.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:17.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:17.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:17.326+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe6280fe900 con 0x7fe614039200 2024-03-16T08:09:17.359 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:17.052+0000 7f75a4981200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:09:17.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:17.626 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:17.257+0000 7f81e9058200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:09:17.626 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:17.403+0000 7f81e9058200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:09:17.843 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:17.750+0000 7f75a4981200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:09:17.843 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:17.842+0000 7f75a4981200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:09:17.975 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:17.749+0000 7f81e9058200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:09:17.975 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:17.838+0000 7f81e9058200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:09:18.109 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:17.989+0000 7f75a4981200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:09:18.109 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.090+0000 7f75a4981200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:09:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-mon[36561]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:18.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:18 smithi049 ceph-mon[31942]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:18.234 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:18.234 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:18.234 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:18.441 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.173+0000 7f75a4981200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:09:18.441 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.441+0000 7f75a4981200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:09:18.474 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:18.037+0000 7f81e9058200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:09:18.474 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:18.227+0000 7f81e9058200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:09:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-mon[36471]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:18.727 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.550+0000 7f75a4981200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:09:18.727 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.635+0000 7f75a4981200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:09:18.727 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.727+0000 7f75a4981200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:09:18.772 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:18.511+0000 7f81e9058200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:09:18.772 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:18.772+0000 7f81e9058200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:09:19.043 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:19.006+0000 7f81e9058200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:09:19.043 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:18.901+0000 7f75a4981200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:20.049 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:19.798+0000 7f81e9058200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:09:20.322 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:19.863+0000 7f75a4981200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:19.991+0000 7f75a4981200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:20.322+0000 7f75a4981200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]: dispatch 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]': finished 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-mon[36561]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T08:09:20.323 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:20.370 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]: dispatch 2024-03-16T08:09:20.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]': finished 2024-03-16T08:09:20.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-mon[36471]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T08:09:20.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:20.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:20 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]: dispatch 2024-03-16T08:09:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:20 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2541503659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e333c72-baee-40c0-a078-372ada0c1511"}]': finished 2024-03-16T08:09:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:20 smithi049 ceph-mon[31942]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T08:09:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:20 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:20.603 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:20.410+0000 7f75a4981200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:09:20.603 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:20.603+0000 7f75a4981200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:09:20.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:20.370+0000 7f81e9058200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:09:20.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:20.467+0000 7f81e9058200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:09:20.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:09:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:09:20.680+0000 7f81e9058200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:09:20.859 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:20.785+0000 7f75a4981200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:09:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:21.053+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7fe61c003b80 con 0x7fe628101260 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:21.066+0000 7f75a4981200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:21.326+0000 7f75a4981200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2339327943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: Standby manager daemon z started 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:09:21.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:21.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2339327943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: Standby manager daemon z started 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:09:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:21 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:21.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2339327943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: Standby manager daemon z started 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:09:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:21 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/3163138486' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:21.609 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:21.558+0000 7f75a4981200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:09:22.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:22 smithi129 ceph-mon[36561]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-16T08:09:22.344 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:22 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:09:22.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:22 smithi154 ceph-mon[36471]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-16T08:09:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:22 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:09:22.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:22 smithi049 ceph-mon[31942]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-16T08:09:22.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:22 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:09:22.609 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:22.343+0000 7f75a4981200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:09:23.217 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:23 smithi129 ceph-mon[36561]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:23.217 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:22.911+0000 7f75a4981200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:09:23.217 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:23.007+0000 7f75a4981200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:09:23.217 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:09:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:09:23.217+0000 7f75a4981200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:09:23.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:23 smithi154 ceph-mon[36471]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:23.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:23 smithi049 ceph-mon[31942]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:24.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:24.072+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fe61c016830 con 0x7fe628101260 2024-03-16T08:09:24.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:24 smithi049 ceph-mon[31942]: Standby manager daemon y started 2024-03-16T08:09:24.163 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:24 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:09:24.163 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:24 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:24.163 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:24 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:09:24.163 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:24 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:24.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:24 smithi129 ceph-mon[36561]: Standby manager daemon y started 2024-03-16T08:09:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:24 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:09:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:24 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:24 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:09:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:24 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:24.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:24 smithi154 ceph-mon[36471]: Standby manager daemon y started 2024-03-16T08:09:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:24 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:09:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:24 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:09:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:24 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:09:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:24 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/1788743314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:09:25.334 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:25 smithi154 ceph-mon[36471]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:25.335 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:25 smithi154 ceph-mon[36471]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-16T08:09:25.335 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:25 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:09:25.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:25 smithi129 ceph-mon[36561]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:25 smithi129 ceph-mon[36561]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-16T08:09:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:25 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:09:25.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:25 smithi049 ceph-mon[31942]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:25 smithi049 ceph-mon[31942]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-16T08:09:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:25 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:09:27.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:27 smithi129 ceph-mon[36561]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:27.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T08:09:27.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:27 smithi154 ceph-mon[36471]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T08:09:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:27.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:27 smithi049 ceph-mon[31942]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T08:09:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:28.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:28 smithi049 ceph-mon[31942]: Deploying daemon osd.0 on smithi049 2024-03-16T08:09:28.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:28 smithi129 ceph-mon[36561]: Deploying daemon osd.0 on smithi049 2024-03-16T08:09:28.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:28 smithi154 ceph-mon[36471]: Deploying daemon osd.0 on smithi049 2024-03-16T08:09:29.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:29 smithi129 ceph-mon[36561]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:29.447 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:29 smithi049 ceph-mon[31942]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:29.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:29 smithi154 ceph-mon[36471]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:31.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:31 smithi129 ceph-mon[36561]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:31.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:31 smithi049 ceph-mon[31942]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:31.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:31 smithi154 ceph-mon[36471]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:33.478 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:33 smithi049 ceph-mon[31942]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:33.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:33.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:33 smithi129 ceph-mon[36561]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:33.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:33 smithi154 ceph-mon[36471]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:33.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:35.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:35.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.790+0000 7fe61a7fc700 1 -- 172.21.15.49:0/3176299347 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe6280fe900 con 0x7fe614039200 2024-03-16T08:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 msgr2=0x7fe61403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 0x7fe61403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe624005950 tx=0x7fe62400b410 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 msgr2=0x7fe6281051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d980 tx=0x7fe61c00dc90 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 shutdown_connections 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe614039200 0x7fe61403b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe628102e00 0x7fe628103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe628101260 0x7fe6281051f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 --2- 172.21.15.49:0/3176299347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe628102460 0x7fe628103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.792+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 >> 172.21.15.49:0/3176299347 conn(0x7fe6280fc820 msgr2=0x7fe6280fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.793+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 shutdown_connections 2024-03-16T08:09:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:09:35.793+0000 7fe62f6ee700 1 -- 172.21.15.49:0/3176299347 wait complete. 2024-03-16T08:09:36.449 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.0.service 2024-03-16T08:09:36.451 INFO:tasks.cephadm:Deploying osd.1 on smithi129 with /dev/vg_nvme/lv_4... 2024-03-16T08:09:36.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T08:09:36.693 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:09:37.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:36 smithi049 ceph-mon[31942]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:37.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:36 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:36 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:36 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:37.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:36 smithi049 ceph-mon[31942]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T08:09:37.043 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:36 smithi129 ceph-mon[36561]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:37.043 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:36 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.043 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:36 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.043 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:36 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:37.043 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:36 smithi129 ceph-mon[36561]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T08:09:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:36 smithi154 ceph-mon[36471]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:36 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:36 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:36 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:36 smithi154 ceph-mon[36471]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T08:09:37.892 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:37 smithi129 ceph-mon[36561]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T08:09:37.892 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:37 smithi129 ceph-mon[36561]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T08:09:37.892 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:37 smithi129 ceph-mon[36561]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:09:37.892 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:37 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:38.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-mon[31942]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T08:09:38.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-mon[31942]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T08:09:38.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-mon[31942]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:09:38.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:38.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0[46677]: 2024-03-16T08:09:37.755+0000 7fa65d556700 -1 osd.0 0 waiting for initial osdmap 2024-03-16T08:09:38.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:09:37 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0[46677]: 2024-03-16T08:09:37.758+0000 7fa659349700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:09:38.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:37 smithi154 ceph-mon[36471]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T08:09:38.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:37 smithi154 ceph-mon[36471]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T08:09:38.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:37 smithi154 ceph-mon[36471]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:09:38.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:37 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:38.471 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T08:09:38.471 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T08:09:38.471 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-03-16T08:09:38.471 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-03-16T08:09:38.471 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129506 s, 810 MB/s 2024-03-16T08:09:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-03-16T08:09:38.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:38 smithi049 ceph-mon[31942]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:38.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:38 smithi049 ceph-mon[31942]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:09:38.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:38 smithi049 ceph-mon[31942]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T08:09:38.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:38.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:38 smithi129 ceph-mon[36561]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:39.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:38 smithi129 ceph-mon[36561]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:09:39.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:38 smithi129 ceph-mon[36561]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T08:09:39.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-03-16T08:09:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:38 smithi154 ceph-mon[36471]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:38 smithi154 ceph-mon[36471]: from='osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:09:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:38 smithi154 ceph-mon[36471]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T08:09:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.262 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] boot 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi049 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-16T08:09:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:39.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:39.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:39.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] boot 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi049 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:40.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:40.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: osd.0 [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] boot 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi049 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:09:40.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 -- 172.21.15.129:0/3614207889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 msgr2=0x7fd5c0101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 --2- 172.21.15.129:0/3614207889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c0101690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0099c0 tx=0x7fd5bc009cd0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 -- 172.21.15.129:0/3614207889 shutdown_connections 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 --2- 172.21.15.129:0/3614207889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102e10 0x7fd5c01096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 --2- 172.21.15.129:0/3614207889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c0101690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 --2- 172.21.15.129:0/3614207889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102470 0x7fd5c01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 -- 172.21.15.129:0/3614207889 >> 172.21.15.129:0/3614207889 conn(0x7fd5c00fc830 msgr2=0x7fd5c00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 -- 172.21.15.129:0/3614207889 shutdown_connections 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.704+0000 7fd5c7243700 1 -- 172.21.15.129:0/3614207889 wait complete. 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 Processor -- start 2024-03-16T08:09:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 -- start start 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102470 0x7fd5c019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102e10 0x7fd5c019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010ce50 con 0x7fd5c0102e10 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010ccd0 con 0x7fd5c0101270 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c7243700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010cfd0 con 0x7fd5c0102470 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34686/0 (socket says 172.21.15.129:34686) 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 -- 172.21.15.129:0/768779246 learned_addr learned my addr 172.21.15.129:0/768779246 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c57e0700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102e10 0x7fd5c019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5b7fff700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102470 0x7fd5c019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 -- 172.21.15.129:0/768779246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102470 msgr2=0x7fd5c019d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102470 0x7fd5c019d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 -- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102e10 msgr2=0x7fd5c019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102e10 0x7fd5c019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.705+0000 7fd5c4fdf700 1 -- 172.21.15.129:0/768779246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c01a01c0 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5c4fdf700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00c010 tx=0x7fd5bc009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc027070 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc004150 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c0197070 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc022d40 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.706+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c01975d0 con 0x7fd5c0101270 2024-03-16T08:09:40.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.707+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c0068f90 con 0x7fd5c0101270 2024-03-16T08:09:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.708+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fd5bc02a070 con 0x7fd5c0101270 2024-03-16T08:09:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.708+0000 7fd5b5ffb700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 0x7fd5a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:09:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.709+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd5bc0ce210 con 0x7fd5c0101270 2024-03-16T08:09:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.709+0000 7fd5b7fff700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 0x7fd5a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:09:40.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.709+0000 7fd5b7fff700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 0x7fd5a80a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5c006bab0 tx=0x7fd5b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:09:40.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.712+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd5bc031070 con 0x7fd5c0101270 2024-03-16T08:09:40.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:40.917+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd5c0062710 con 0x7fd5a80a2800 2024-03-16T08:09:41.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:40 smithi049 ceph-mon[31942]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:41.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:40 smithi049 ceph-mon[31942]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T08:09:41.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:40 smithi129 ceph-mon[36561]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:41.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:40 smithi129 ceph-mon[36561]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T08:09:41.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:40 smithi154 ceph-mon[36471]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:09:41.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:40 smithi154 ceph-mon[36471]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T08:09:42.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:41 smithi049 ceph-mon[31942]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:42.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:42.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:42.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:42.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:41 smithi129 ceph-mon[36561]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:42.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:42.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:42.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:42.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:41 smithi154 ceph-mon[36471]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:09:42.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:09:42.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:09:42.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:42.859 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:42 smithi129 ceph-mon[36561]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:43.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:42 smithi049 ceph-mon[31942]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:43.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:42 smithi154 ceph-mon[36471]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:44.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:43 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/479050030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:43 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:43 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]': finished 2024-03-16T08:09:44.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:43 smithi049 ceph-mon[31942]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T08:09:44.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:09:44.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:43 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/479050030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:43 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:43 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]': finished 2024-03-16T08:09:44.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:43 smithi129 ceph-mon[36561]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T08:09:44.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:09:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:43 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/479050030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:43 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]: dispatch 2024-03-16T08:09:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:43 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ddf11d-d24d-4221-9317-a912d69ff440"}]': finished 2024-03-16T08:09:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:43 smithi154 ceph-mon[36471]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T08:09:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:09:45.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:44 smithi049 ceph-mon[31942]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:45.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:44 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/325703785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:45.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:44 smithi129 ceph-mon[36561]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:45.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:44 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/325703785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:45.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:44 smithi154 ceph-mon[36471]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:45.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:44 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/325703785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:09:47.031 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:46 smithi129 ceph-mon[36561]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:47.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:46 smithi049 ceph-mon[31942]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:47.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:46 smithi154 ceph-mon[36471]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:49.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:48 smithi049 ceph-mon[31942]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:49.054 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:48 smithi129 ceph-mon[36561]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:49.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:48 smithi154 ceph-mon[36471]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:51.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:50 smithi049 ceph-mon[31942]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:51.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T08:09:51.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:51.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:50 smithi049 ceph-mon[31942]: Deploying daemon osd.1 on smithi129 2024-03-16T08:09:51.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:50 smithi129 ceph-mon[36561]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:51.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T08:09:51.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:51.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:50 smithi129 ceph-mon[36561]: Deploying daemon osd.1 on smithi129 2024-03-16T08:09:51.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:50 smithi154 ceph-mon[36471]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:51.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T08:09:51.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:09:51.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:50 smithi154 ceph-mon[36471]: Deploying daemon osd.1 on smithi129 2024-03-16T08:09:53.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:52 smithi049 ceph-mon[31942]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:53.054 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:52 smithi129 ceph-mon[36561]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:53.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:52 smithi154 ceph-mon[36471]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:54.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:54 smithi129 ceph-mon[36561]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:55.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:54 smithi154 ceph-mon[36471]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:55.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:54 smithi049 ceph-mon[31942]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:56 smithi154 ceph-mon[36471]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:56 smithi049 ceph-mon[31942]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:09:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:56 smithi129 ceph-mon[36561]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:58.282 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 1 on host 'smithi129' 2024-03-16T08:09:58.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.281+0000 7fd5b5ffb700 1 -- 172.21.15.129:0/768779246 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5c0062710 con 0x7fd5a80a2800 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 msgr2=0x7fd5a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 0x7fd5a80a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5c006bab0 tx=0x7fd5b000b410 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 msgr2=0x7fd5c006b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00c010 tx=0x7fd5bc009510 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 shutdown_connections 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd5a80a2800 0x7fd5a80a4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.284+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5c0102e10 0x7fd5c019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.285+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0101270 0x7fd5c006b550 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.285+0000 7fd5c7243700 1 --2- 172.21.15.129:0/768779246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c0102470 0x7fd5c019d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.285+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 >> 172.21.15.129:0/768779246 conn(0x7fd5c00fc830 msgr2=0x7fd5c01057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.285+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 shutdown_connections 2024-03-16T08:09:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:09:58.285+0000 7fd5c7243700 1 -- 172.21.15.129:0/768779246 wait complete. 2024-03-16T08:09:59.105 DEBUG:teuthology.orchestra.run.smithi129:osd.1> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.1.service 2024-03-16T08:09:59.107 INFO:tasks.cephadm:Deploying osd.2 on smithi154 with /dev/vg_nvme/lv_4... 2024-03-16T08:09:59.108 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T08:09:59.335 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-mon[36561]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:59.336 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.336 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.336 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.336 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.355 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:09:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:59 smithi154 ceph-mon[36471]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:09:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:59 smithi049 ceph-mon[31942]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:09:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:09:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:09:59.610 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:09:59 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:09:59.334+0000 7f949f4327c0 -1 osd.1 0 log_to_monitors true 2024-03-16T08:10:00.324 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:00 smithi129 ceph-mon[36561]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.324 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:00 smithi129 ceph-mon[36561]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.324 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:00 smithi129 ceph-mon[36561]: overall HEALTH_OK 2024-03-16T08:10:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:00 smithi154 ceph-mon[36471]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:00 smithi154 ceph-mon[36471]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:00 smithi154 ceph-mon[36471]: overall HEALTH_OK 2024-03-16T08:10:00.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:00 smithi049 ceph-mon[31942]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:00 smithi049 ceph-mon[31942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:10:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:00 smithi049 ceph-mon[31942]: overall HEALTH_OK 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112637 s, 931 MB/s 2024-03-16T08:10:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:01 smithi049 ceph-mon[31942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-mon[36561]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: from='osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:01.676 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:01 smithi154 ceph-mon[36471]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:10:01.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-03-16T08:10:02.110 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:10:01.718+0000 7f9494a97700 -1 osd.1 0 waiting for initial osdmap 2024-03-16T08:10:02.110 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:10:01.723+0000 7f948c882700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:10:02.239 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:10:02.464 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:02 smithi129 ceph-mon[36561]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:10:02.465 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:02 smithi129 ceph-mon[36561]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T08:10:02.465 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.465 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.465 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:02 smithi129 ceph-mon[36561]: from='osd.1 ' entity='osd.1' 2024-03-16T08:10:02.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:02 smithi049 ceph-mon[31942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:10:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:02 smithi049 ceph-mon[31942]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T08:10:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:02 smithi049 ceph-mon[31942]: from='osd.1 ' entity='osd.1' 2024-03-16T08:10:02.635 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:02 smithi154 ceph-mon[36471]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:10:02.635 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:02 smithi154 ceph-mon[36471]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T08:10:02.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:02.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:02 smithi154 ceph-mon[36471]: from='osd.1 ' entity='osd.1' 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] boot 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.332 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] boot 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: osd.1 [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] boot 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:10:03.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 -- 172.21.15.154:0/3276908076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944103500 msgr2=0x7fd944103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/3276908076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944103500 0x7fd944103960 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd938009990 tx=0x7fd938009ca0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 -- 172.21.15.154:0/3276908076 shutdown_connections 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/3276908076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103ea0 0x7fd94410a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/3276908076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944102300 0x7fd944102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/3276908076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944103500 0x7fd944103960 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 -- 172.21.15.154:0/3276908076 >> 172.21.15.154:0/3276908076 conn(0x7fd9440fd8e0 msgr2=0x7fd9440ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 -- 172.21.15.154:0/3276908076 shutdown_connections 2024-03-16T08:10:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.819+0000 7fd94a1a0700 1 -- 172.21.15.154:0/3276908076 wait complete. 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.820+0000 7fd94a1a0700 1 Processor -- start 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.820+0000 7fd94a1a0700 1 -- start start 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103500 0x7fd9441989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944103ea0 0x7fd94419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410ded0 con 0x7fd944103ea0 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410dd50 con 0x7fd944103500 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410e050 con 0x7fd944102300 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40432/0 (socket says 172.21.15.154:40432) 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 -- 172.21.15.154:0/4107359046 learned_addr learned my addr 172.21.15.154:0/4107359046 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd942ffd700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103500 0x7fd9441989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd943fff700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944103ea0 0x7fd94419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 -- 172.21.15.154:0/4107359046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103500 msgr2=0x7fd9441989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103500 0x7fd9441989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 -- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944103ea0 msgr2=0x7fd94419fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944103ea0 0x7fd94419fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 -- 172.21.15.154:0/4107359046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441a00f0 con 0x7fd944102300 2024-03-16T08:10:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd9437fe700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd93400d9b0 tx=0x7fd93400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934017030 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd934004500 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9441a03e0 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003970 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.821+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441a0920 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.822+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd944068f90 con 0x7fd944102300 2024-03-16T08:10:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.823+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fd93401a070 con 0x7fd944102300 2024-03-16T08:10:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.824+0000 7fd940ff9700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 0x7fd92c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.824+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fd9340ccde0 con 0x7fd944102300 2024-03-16T08:10:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.824+0000 7fd942ffd700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 0x7fd92c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.825+0000 7fd942ffd700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 0x7fd92c0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd93800b5c0 tx=0x7fd938003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:03.828+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd93400e350 con 0x7fd944102300 2024-03-16T08:10:04.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:04.032+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd944062710 con 0x7fd92c0a2800 2024-03-16T08:10:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:04 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi129 2024-03-16T08:10:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:04 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 17080M 2024-03-16T08:10:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:04.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:04 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi129 2024-03-16T08:10:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:04 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 17080M 2024-03-16T08:10:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:04.630 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:04 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi129 2024-03-16T08:10:04.630 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:04 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 17080M 2024-03-16T08:10:04.631 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:04.631 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:04.631 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:05.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:05 smithi154 ceph-mon[36471]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:05.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:05 smithi154 ceph-mon[36471]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:05.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:05 smithi154 ceph-mon[36471]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T08:10:05.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:05 smithi049 ceph-mon[31942]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:05.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:05 smithi049 ceph-mon[31942]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:05.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:05 smithi049 ceph-mon[31942]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T08:10:05.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:05 smithi129 ceph-mon[36561]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:05 smithi129 ceph-mon[36561]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:05 smithi129 ceph-mon[36561]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/3251993917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]': finished 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:07 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/270360695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:07.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/3251993917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]': finished 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:07 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/270360695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/3251993917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]: dispatch 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77bcc416-5d33-4e76-bd0b-7b12aceb1a27"}]': finished 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T08:10:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:07.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:07 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/270360695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:09.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:09 smithi129 ceph-mon[36561]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:09.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:09 smithi154 ceph-mon[36471]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:09.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:09 smithi049 ceph-mon[31942]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:11.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:11 smithi154 ceph-mon[36471]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:11.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:11 smithi129 ceph-mon[36561]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:11.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:11 smithi049 ceph-mon[31942]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:13.474 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:13 smithi154 ceph-mon[36471]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:13.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:13 smithi129 ceph-mon[36561]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:13.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:13 smithi049 ceph-mon[31942]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:14.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T08:10:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:14.618 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T08:10:14.619 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:14.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T08:10:14.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:15.459 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:15 smithi154 ceph-mon[36471]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:15.459 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:15 smithi154 ceph-mon[36471]: Deploying daemon osd.2 on smithi154 2024-03-16T08:10:15.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:15 smithi129 ceph-mon[36561]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:15.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:15 smithi129 ceph-mon[36561]: Deploying daemon osd.2 on smithi154 2024-03-16T08:10:15.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:15 smithi049 ceph-mon[31942]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:15.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:15 smithi049 ceph-mon[31942]: Deploying daemon osd.2 on smithi154 2024-03-16T08:10:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:17 smithi154 ceph-mon[36471]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:17.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:17 smithi129 ceph-mon[36561]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:17.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:17 smithi049 ceph-mon[31942]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:19 smithi154 ceph-mon[36471]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:19.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:19 smithi129 ceph-mon[36561]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:19.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:19.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:19 smithi049 ceph-mon[31942]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:19.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:21.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:21 smithi154 ceph-mon[36471]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:21.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:21 smithi129 ceph-mon[36561]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:21.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:21 smithi049 ceph-mon[31942]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:22.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.653+0000 7fd940ff9700 1 -- 172.21.15.154:0/4107359046 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd944062710 con 0x7fd92c0a2800 2024-03-16T08:10:22.653 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 2 on host 'smithi154' 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 msgr2=0x7fd92c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 0x7fd92c0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd93800b5c0 tx=0x7fd938003960 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 msgr2=0x7fd944198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd93400d9b0 tx=0x7fd93400dcc0 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 shutdown_connections 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd92c0a2800 0x7fd92c0a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd944103ea0 0x7fd94419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd944102300 0x7fd944198490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 --2- 172.21.15.154:0/4107359046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd944103500 0x7fd9441989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.655+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 >> 172.21.15.154:0/4107359046 conn(0x7fd9440fd8e0 msgr2=0x7fd944106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.656+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 shutdown_connections 2024-03-16T08:10:22.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:10:22.656+0000 7fd94a1a0700 1 -- 172.21.15.154:0/4107359046 wait complete. 2024-03-16T08:10:23.283 DEBUG:teuthology.orchestra.run.smithi154:osd.2> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.2.service 2024-03-16T08:10:23.285 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_3... 2024-03-16T08:10:23.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T08:10:23.548 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:10:23.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:23 smithi129 ceph-mon[36561]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:23.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:23 smithi129 ceph-mon[36561]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:23 smithi129 ceph-mon[36561]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:23.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:23 smithi049 ceph-mon[31942]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:23.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:23 smithi049 ceph-mon[31942]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:23.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:23 smithi049 ceph-mon[31942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:23.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:23 smithi154 ceph-mon[36471]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:23.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:23.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:23 smithi154 ceph-mon[36471]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:23.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:23 smithi154 ceph-mon[36471]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:10:24.607 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:24.608 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:24 smithi049 ceph-mon[31942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:24 smithi129 ceph-mon[36561]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:24 smithi154 ceph-mon[36471]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:10:25.225 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2[41621]: 2024-03-16T08:10:25.065+0000 7fc535ef3700 -1 osd.2 0 waiting for initial osdmap 2024-03-16T08:10:25.225 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2[41621]: 2024-03-16T08:10:25.069+0000 7fc52d4dd700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00861311 s, 1.2 GB/s 2024-03-16T08:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:25 smithi049 ceph-mon[31942]: from='osd.2 ' entity='osd.2' 2024-03-16T08:10:25.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:25.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:10:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T08:10:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:25 smithi129 ceph-mon[36561]: from='osd.2 ' entity='osd.2' 2024-03-16T08:10:25.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:10:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T08:10:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:25 smithi154 ceph-mon[36471]: from='osd.2 ' entity='osd.2' 2024-03-16T08:10:26.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-03-16T08:10:26.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:10:26.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:10:26.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:26.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] boot 2024-03-16T08:10:26.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T08:10:26.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:26.377 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] boot 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T08:10:26.713 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: osd.2 [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] boot 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T08:10:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:10:27.456 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:27.456 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.457 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:27 smithi049 ceph-mon[31942]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:27 smithi154 ceph-mon[36471]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:27 smithi129 ceph-mon[36561]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T08:10:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 -- 172.21.15.49:0/1308138838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103520 msgr2=0x7fe8b0103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/1308138838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103520 0x7fe8b0103980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8980099c0 tx=0x7fe898009cd0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 -- 172.21.15.49:0/1308138838 shutdown_connections 2024-03-16T08:10:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/1308138838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0103ec0 0x7fe8b010a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/1308138838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0102320 0x7fe8b0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/1308138838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103520 0x7fe8b0103980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 -- 172.21.15.49:0/1308138838 >> 172.21.15.49:0/1308138838 conn(0x7fe8b00fd8e0 msgr2=0x7fe8b00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:10:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 -- 172.21.15.49:0/1308138838 shutdown_connections 2024-03-16T08:10:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.093+0000 7fe8b5f38700 1 -- 172.21.15.49:0/1308138838 wait complete. 2024-03-16T08:10:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 Processor -- start 2024-03-16T08:10:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 -- start start 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0102320 0x7fe8b01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 0x7fe8b0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010dfb0 con 0x7fe8b0103ec0 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010de30 con 0x7fe8b0103520 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.094+0000 7fe8b5f38700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010e130 con 0x7fe8b0102320 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37426/0 (socket says 172.21.15.49:37426) 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 -- 172.21.15.49:0/3599658805 learned_addr learned my addr 172.21.15.49:0/3599658805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8af7fe700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0102320 0x7fe8b01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8aeffd700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 0x7fe8b0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 -- 172.21.15.49:0/3599658805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0102320 msgr2=0x7fe8b01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0102320 0x7fe8b01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 -- 172.21.15.49:0/3599658805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 msgr2=0x7fe8b0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 0x7fe8b0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 -- 172.21.15.49:0/3599658805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01a0200 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8aeffd700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 0x7fe8b0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8affff700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007f10 tx=0x7fe8a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8004bb0 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b01a04f0 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a8004d50 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005a60 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.095+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a0b00 con 0x7fe8b0103ec0 2024-03-16T08:10:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.096+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b0068f90 con 0x7fe8b0103ec0 2024-03-16T08:10:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.098+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fe8a801a020 con 0x7fe8b0103ec0 2024-03-16T08:10:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.099+0000 7fe8acff9700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 0x7fe89c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.099+0000 7fe8af7fe700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 0x7fe89c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.099+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fe8a80cd1d0 con 0x7fe8b0103ec0 2024-03-16T08:10:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.100+0000 7fe8af7fe700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 0x7fe89c0a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0103380 tx=0x7fe8a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.103+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe8a8010070 con 0x7fe8b0103ec0 2024-03-16T08:10:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:28.318+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe8b0062710 con 0x7fe89c0a2800 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi154 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 17080M 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:28.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi154 2024-03-16T08:10:28.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 17080M 2024-03-16T08:10:28.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:28.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:28.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:28.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:28.859 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi154 2024-03-16T08:10:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 17080M 2024-03-16T08:10:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:29.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:29 smithi049 ceph-mon[31942]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:29.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:29 smithi049 ceph-mon[31942]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:29.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:29.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:29 smithi049 ceph-mon[31942]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T08:10:29.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:29 smithi154 ceph-mon[36471]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:29 smithi154 ceph-mon[36471]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:29 smithi154 ceph-mon[36471]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T08:10:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:29 smithi129 ceph-mon[36561]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:29 smithi129 ceph-mon[36561]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:29 smithi129 ceph-mon[36561]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T08:10:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:10:30.036 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:29 smithi049 sudo[51268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:30.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:29 smithi049 sudo[51268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:30.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:29 smithi049 sudo[51268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:30.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:29 smithi049 sudo[51268]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:30.386+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe8a8097810 con 0x7fe8b0103ec0 2024-03-16T08:10:30.653 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:30.653 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:30 smithi129 ceph-mon[36561]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T08:10:30.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:30.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:30 smithi154 ceph-mon[36471]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T08:10:30.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:10:30.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:30 smithi049 ceph-mon[31942]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T08:10:30.787 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:30 smithi049 sudo[51281]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:30.787 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:30 smithi049 sudo[51281]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:30.787 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:30 smithi049 sudo[51281]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:30.787 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:10:30 smithi049 sudo[51281]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:30.907 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:30.907 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:30.907 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:30.908 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45029]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:31.360 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45032]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:31.360 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:31.360 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45032]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:31.360 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:10:30 smithi129 sudo[45032]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:31.437 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45541]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:31.437 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45541]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:31.437 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45541]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:31.437 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45541]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:31.437 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:31.437 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T08:10:31.437 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-16T08:10:31.437 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:31.438 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]: dispatch 2024-03-16T08:10:31.438 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]': finished 2024-03-16T08:10:31.438 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T08:10:31.438 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:31.438 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:31.725 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45544]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:31.725 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45544]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:31.725 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45544]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:31.725 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:10:31 smithi154 sudo[45544]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:31.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:31.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]: dispatch 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]': finished 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]: dispatch 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1947276435' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6b81081-263e-4e72-8499-aea94ed43b50"}]': finished 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:31.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:32.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 sudo[51360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 sudo[51360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 sudo[51360]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:31 smithi049 sudo[51360]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:32 smithi049 sudo[51363]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:32 smithi049 sudo[51363]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:32 smithi049 sudo[51363]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:32 smithi049 sudo[51363]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:32.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:32.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:32.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45547]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:32.852 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45547]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:33.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45550]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:32 smithi154 sudo[45550]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:33.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:10:33.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:33.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45035]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:33.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45035]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:33.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45038]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45038]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 sudo[45038]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2845729771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2845729771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:33.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2845729771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:34 smithi129 ceph-mon[36561]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:34 smithi154 ceph-mon[36471]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:34.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:10:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:34 smithi049 ceph-mon[31942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:10:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:35 smithi129 ceph-mon[36561]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:35.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:35 smithi049 ceph-mon[31942]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:35 smithi154 ceph-mon[36471]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:37 smithi129 ceph-mon[36561]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:37.627 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:37 smithi049 ceph-mon[31942]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:37.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:37 smithi154 ceph-mon[36471]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:39.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:39 smithi049 ceph-mon[31942]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:39.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T08:10:39.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:39.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:39 smithi154 ceph-mon[36471]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:39.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T08:10:39.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:39 smithi129 ceph-mon[36561]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T08:10:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:40.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:40 smithi049 ceph-mon[31942]: Deploying daemon osd.3 on smithi049 2024-03-16T08:10:40.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:40 smithi154 ceph-mon[36471]: Deploying daemon osd.3 on smithi049 2024-03-16T08:10:40.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:40 smithi129 ceph-mon[36561]: Deploying daemon osd.3 on smithi049 2024-03-16T08:10:41.712 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:41 smithi049 ceph-mon[31942]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-16T08:10:41.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:41 smithi154 ceph-mon[36471]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-16T08:10:41.859 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:41 smithi129 ceph-mon[36561]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-16T08:10:43.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:43 smithi049 ceph-mon[31942]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-16T08:10:43.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:43 smithi154 ceph-mon[36471]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-16T08:10:43.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:43 smithi129 ceph-mon[36561]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-16T08:10:45.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:45 smithi049 ceph-mon[31942]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-16T08:10:45.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:45 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:45 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:45 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:45.662 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:45 smithi129 ceph-mon[36561]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-16T08:10:45.662 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:45 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.662 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:45 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.662 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:45 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:45 smithi154 ceph-mon[36471]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-16T08:10:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:45 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:45 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:45 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:47.514 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.124+0000 7fe8acff9700 1 -- 172.21.15.49:0/3599658805 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8b0062710 con 0x7fe89c0a2800 2024-03-16T08:10:48.124 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 msgr2=0x7fe89c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 0x7fe89c0a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0103380 tx=0x7fe8a0009190 comp rx=0 tx=0).stop 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 msgr2=0x7fe8b019fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007f10 tx=0x7fe8a800d3b0 comp rx=0 tx=0).stop 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 shutdown_connections 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe89c0a2800 0x7fe89c0a4cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b0103ec0 0x7fe8b019fb60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8b0102320 0x7fe8b01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 --2- 172.21.15.49:0/3599658805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0103520 0x7fe8b0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.126+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 >> 172.21.15.49:0/3599658805 conn(0x7fe8b00fd8e0 msgr2=0x7fe8b0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:10:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.127+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 shutdown_connections 2024-03-16T08:10:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:10:48.127+0000 7fe8b5f38700 1 -- 172.21.15.49:0/3599658805 wait complete. 2024-03-16T08:10:48.769 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.3.service 2024-03-16T08:10:48.772 INFO:tasks.cephadm:Deploying osd.4 on smithi129 with /dev/vg_nvme/lv_3... 2024-03-16T08:10:48.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T08:10:49.018 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:10:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:49 smithi129 ceph-mon[36561]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:49 smithi129 ceph-mon[36561]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:10:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:49 smithi154 ceph-mon[36471]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:49 smithi154 ceph-mon[36471]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:10:49.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:49 smithi049 ceph-mon[31942]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T08:10:49.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:49.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:10:49.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:49 smithi049 ceph-mon[31942]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:10:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:50 smithi154 ceph-mon[36471]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T08:10:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:50 smithi154 ceph-mon[36471]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T08:10:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:50 smithi154 ceph-mon[36471]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:10:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-mon[31942]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T08:10:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-mon[31942]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T08:10:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-mon[31942]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:10:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:50.787 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3[53378]: 2024-03-16T08:10:50.388+0000 7f1bafa5c700 -1 osd.3 0 waiting for initial osdmap 2024-03-16T08:10:50.787 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:10:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3[53378]: 2024-03-16T08:10:50.393+0000 7f1bab84f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:10:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:50 smithi129 ceph-mon[36561]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T08:10:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:50 smithi129 ceph-mon[36561]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T08:10:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:50 smithi129 ceph-mon[36561]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:10:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111881 s, 937 MB/s 2024-03-16T08:10:50.945 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-03-16T08:10:51.533 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:51 smithi129 ceph-mon[36561]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:51.533 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:51 smithi129 ceph-mon[36561]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:10:51.534 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:51 smithi129 ceph-mon[36561]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T08:10:51.534 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:51 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.534 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:51 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:51 smithi049 ceph-mon[31942]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:51.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:51 smithi049 ceph-mon[31942]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:10:51.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:51 smithi049 ceph-mon[31942]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T08:10:51.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:51 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:51 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-03-16T08:10:51.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:51 smithi154 ceph-mon[36471]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:51 smithi154 ceph-mon[36471]: from='osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:10:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:51 smithi154 ceph-mon[36471]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T08:10:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:51 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:51 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:51.806 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] boot 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi049 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-16T08:10:52.607 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:52.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:52.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] boot 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi049 2024-03-16T08:10:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:52.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: osd.3 [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] boot 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi049 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-16T08:10:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:52.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:52.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:10:52.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:10:53.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- 172.21.15.129:0/3135123183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 msgr2=0x7fdc34106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:53.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 --2- 172.21.15.129:0/3135123183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc34106340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b0a0 tx=0x7fdc2c00b3b0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- 172.21.15.129:0/3135123183 shutdown_connections 2024-03-16T08:10:53.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 --2- 172.21.15.129:0/3135123183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc34106340 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 --2- 172.21.15.129:0/3135123183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 0x7fdc34102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 --2- 172.21.15.129:0/3135123183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 0x7fdc34105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- 172.21.15.129:0/3135123183 >> 172.21.15.129:0/3135123183 conn(0x7fdc340fc7c0 msgr2=0x7fdc340febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- 172.21.15.129:0/3135123183 shutdown_connections 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- 172.21.15.129:0/3135123183 wait complete. 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 Processor -- start 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.371+0000 7fdc3c074700 1 -- start start 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 0x7fdc3419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 0x7fdc3419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34114470 con 0x7fdc34100b80 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341142f0 con 0x7fdc34105ec0 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341145f0 con 0x7fdc341034f0 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:53.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48676/0 (socket says 172.21.15.129:48676) 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 -- 172.21.15.129:0/629893545 learned_addr learned my addr 172.21.15.129:0/629893545 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc39e10700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 0x7fdc3419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3960f700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 0x7fdc3419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 -- 172.21.15.129:0/629893545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 msgr2=0x7fdc3419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 0x7fdc3419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 -- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 msgr2=0x7fdc3419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 0x7fdc3419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 -- 172.21.15.129:0/629893545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc341a66b0 con 0x7fdc34105ec0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3a611700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c007f00 tx=0x7fdc2c004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c021070 con 0x7fdc34105ec0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc341a6940 con 0x7fdc34105ec0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c007c00 con 0x7fdc34105ec0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.372+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c005460 con 0x7fdc34105ec0 2024-03-16T08:10:53.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.373+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a6df0 con 0x7fdc34105ec0 2024-03-16T08:10:53.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.373+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc34068f90 con 0x7fdc34105ec0 2024-03-16T08:10:53.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.374+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdc2c024070 con 0x7fdc34105ec0 2024-03-16T08:10:53.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.375+0000 7fdc2affd700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 0x7fdc200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:10:53.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.375+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fdc2c00f020 con 0x7fdc34105ec0 2024-03-16T08:10:53.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.375+0000 7fdc39e10700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 0x7fdc200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:10:53.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.376+0000 7fdc39e10700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 0x7fdc200a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:10:53.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.380+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc2c02b020 con 0x7fdc34105ec0 2024-03-16T08:10:53.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:10:53.587+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc34062710 con 0x7fdc200a2890 2024-03-16T08:10:53.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:53 smithi129 ceph-mon[36561]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T08:10:53.724 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:53 smithi154 ceph-mon[36471]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T08:10:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:53 smithi049 ceph-mon[31942]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:54 smithi154 ceph-mon[36471]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:54 smithi129 ceph-mon[36561]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:10:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:54 smithi049 ceph-mon[31942]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T08:10:56.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:55 smithi129 ceph-mon[36561]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T08:10:56.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:55 smithi154 ceph-mon[36471]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T08:10:56.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:55 smithi049 ceph-mon[31942]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/1360068905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]': finished 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T08:10:57.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/1360068905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]': finished 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T08:10:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/1360068905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]: dispatch 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99b2da5-a3b6-423d-9884-497aae8108de"}]': finished 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T08:10:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:10:58.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:57 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/126929625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:58.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:57 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/126929625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:58.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:57 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/126929625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:10:59.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:10:58 smithi129 ceph-mon[36561]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:59.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:10:58 smithi154 ceph-mon[36471]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:10:59.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:10:58 smithi049 ceph-mon[31942]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:01.195 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:00 smithi129 ceph-mon[36561]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:11:01.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:00 smithi154 ceph-mon[36471]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:11:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:00 smithi049 ceph-mon[31942]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:11:03.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:02 smithi129 ceph-mon[36561]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-16T08:11:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:02 smithi154 ceph-mon[36471]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-16T08:11:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:02 smithi049 ceph-mon[31942]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-16T08:11:04.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T08:11:04.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T08:11:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:04.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T08:11:04.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:05.073 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:04 smithi129 ceph-mon[36561]: Deploying daemon osd.4 on smithi129 2024-03-16T08:11:05.073 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:04 smithi129 ceph-mon[36561]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-16T08:11:05.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:04 smithi154 ceph-mon[36471]: Deploying daemon osd.4 on smithi129 2024-03-16T08:11:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:04 smithi154 ceph-mon[36471]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-16T08:11:05.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:04 smithi049 ceph-mon[31942]: Deploying daemon osd.4 on smithi129 2024-03-16T08:11:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:04 smithi049 ceph-mon[31942]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-16T08:11:07.054 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:06 smithi129 ceph-mon[36561]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:11:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:06 smithi154 ceph-mon[36471]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:11:07.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:06 smithi049 ceph-mon[31942]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:11:09.148 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:08 smithi129 ceph-mon[36561]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-16T08:11:09.149 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.149 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.149 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:08 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:08 smithi154 ceph-mon[36471]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-16T08:11:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:08 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:08 smithi049 ceph-mon[31942]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-16T08:11:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:08 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:10 smithi154 ceph-mon[36471]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-16T08:11:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:10 smithi049 ceph-mon[31942]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-16T08:11:11.294 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:10 smithi129 ceph-mon[36561]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-16T08:11:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.002+0000 7fdc2affd700 1 -- 172.21.15.129:0/629893545 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc34062710 con 0x7fdc200a2890 2024-03-16T08:11:12.003 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 4 on host 'smithi129' 2024-03-16T08:11:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 msgr2=0x7fdc200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 0x7fdc200a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc3000b410 comp rx=0 tx=0).stop 2024-03-16T08:11:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 msgr2=0x7fdc341a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c007f00 tx=0x7fdc2c004e40 comp rx=0 tx=0).stop 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 shutdown_connections 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdc200a2890 0x7fdc200a4d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc34105ec0 0x7fdc341a6010 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc34100b80 0x7fdc3419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 --2- 172.21.15.129:0/629893545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc341034f0 0x7fdc3419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 >> 172.21.15.129:0/629893545 conn(0x7fdc340fc7c0 msgr2=0x7fdc34101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 shutdown_connections 2024-03-16T08:11:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:11:12.005+0000 7fdc3c074700 1 -- 172.21.15.129:0/629893545 wait complete. 2024-03-16T08:11:12.619 DEBUG:teuthology.orchestra.run.smithi129:osd.4> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.4.service 2024-03-16T08:11:12.622 INFO:tasks.cephadm:Deploying osd.5 on smithi154 with /dev/vg_nvme/lv_3... 2024-03-16T08:11:12.622 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T08:11:12.868 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:11:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:12 smithi129 ceph-mon[36561]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:13.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:12 smithi154 ceph-mon[36471]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:13.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:13.286 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:12 smithi049 ceph-mon[31942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:11:14.409 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:14 smithi154 ceph-mon[36471]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T08:11:14.409 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:14 smithi154 ceph-mon[36471]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T08:11:14.409 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:14.409 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:14 smithi154 ceph-mon[36471]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.410 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:14 smithi154 ceph-mon[36471]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:14 smithi049 ceph-mon[31942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T08:11:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:14 smithi049 ceph-mon[31942]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T08:11:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:14 smithi049 ceph-mon[31942]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:14 smithi049 ceph-mon[31942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-mon[36561]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T08:11:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-mon[36561]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T08:11:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-mon[36561]: from='osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-mon[36561]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:11:14.956 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:11:14.684+0000 7f00aba55700 -1 osd.4 0 waiting for initial osdmap 2024-03-16T08:11:14.957 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:11:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:11:14.690+0000 7f00a3840700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112284 s, 934 MB/s 2024-03-16T08:11:14.980 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:15 smithi049 ceph-mon[31942]: from='osd.4 ' entity='osd.4' 2024-03-16T08:11:15.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:15.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:11:15.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T08:11:15.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:15 smithi129 ceph-mon[36561]: from='osd.4 ' entity='osd.4' 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:15.627 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:15 smithi154 ceph-mon[36471]: from='osd.4 ' entity='osd.4' 2024-03-16T08:11:15.655 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-03-16T08:11:15.893 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:11:16.280 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] boot 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:16.281 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.282 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.282 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:16 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] boot 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:16 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: osd.4 [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] boot 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:16.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:16 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:17.379 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:17 smithi154 ceph-mon[36471]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:17.379 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:17 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi129 2024-03-16T08:11:17.379 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:17 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 8540M 2024-03-16T08:11:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:17 smithi049 ceph-mon[31942]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:17 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi129 2024-03-16T08:11:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:17 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 8540M 2024-03-16T08:11:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:17 smithi129 ceph-mon[36561]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T08:11:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:17 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi129 2024-03-16T08:11:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:17 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 8540M 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.598+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/1256397675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34102da0 msgr2=0x7f1b34109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.598+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/1256397675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34102da0 0x7f1b34109650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b0a0 tx=0x7f1b2c00b3b0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/1256397675 shutdown_connections 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/1256397675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34102da0 0x7f1b34109650 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/1256397675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34101200 0x7f1b34101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/1256397675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102400 0x7f1b34102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/1256397675 >> 172.21.15.154:0/1256397675 conn(0x7f1b340fc7c0 msgr2=0x7f1b340febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/1256397675 shutdown_connections 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/1256397675 wait complete. 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 Processor -- start 2024-03-16T08:11:17.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- start start 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 0x7f1b34198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102da0 0x7f1b3419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410df30 con 0x7f1b34102da0 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410ddb0 con 0x7f1b34102400 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3c5f0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410e0b0 con 0x7f1b34101200 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.599+0000 7f1b3a38c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56836/0 (socket says 172.21.15.154:56836) 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 -- 172.21.15.154:0/2674559966 learned_addr learned my addr 172.21.15.154:0/2674559966 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b39b8b700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 0x7f1b34198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3ab8d700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102da0 0x7f1b3419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 -- 172.21.15.154:0/2674559966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 msgr2=0x7f1b34198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 0x7f1b34198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 -- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102da0 msgr2=0x7f1b3419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102da0 0x7f1b3419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 -- 172.21.15.154:0/2674559966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b341a00b0 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b39b8b700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 0x7f1b34198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3a38c700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c9a0 tx=0x7f1b3000cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30004220 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30007d40 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b341a03a0 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000f450 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.600+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a0930 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.601+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b34068f90 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.602+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1b30007860 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.603+0000 7f1b2b7fe700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 0x7f1b200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.603+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f1b300cd920 con 0x7f1b34101200 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.603+0000 7f1b39b8b700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 0x7f1b200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:17.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.603+0000 7f1b39b8b700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 0x7f1b200a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009cb0 tx=0x7f1b24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:11:17.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.607+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b30014030 con 0x7f1b34101200 2024-03-16T08:11:17.812 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:17.812+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1b34062710 con 0x7f1b200a2910 2024-03-16T08:11:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:18 smithi049 ceph-mon[31942]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T08:11:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:18.599 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:18 smithi154 ceph-mon[36471]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T08:11:18.599 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:18.599 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:18.599 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:18 smithi129 ceph-mon[36561]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T08:11:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:19 smithi129 ceph-mon[36561]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:19 smithi129 ceph-mon[36561]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:19.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:19 smithi154 ceph-mon[36471]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:19.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:19 smithi154 ceph-mon[36471]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:19.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:19 smithi049 ceph-mon[31942]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:19.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:19 smithi049 ceph-mon[31942]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/3583716972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]': finished 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:21 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/2795376828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/3583716972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]': finished 2024-03-16T08:11:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T08:11:21.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:21.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:21 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/2795376828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/3583716972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]: dispatch 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5563fc-b860-41fd-98a7-5cabd7bb92c9"}]': finished 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:21 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/2795376828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:23 smithi049 ceph-mon[31942]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:23 smithi129 ceph-mon[36561]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:23.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:23 smithi154 ceph-mon[36471]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:25.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:25 smithi154 ceph-mon[36471]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:25 smithi049 ceph-mon[31942]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:25 smithi129 ceph-mon[36561]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:27.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:27 smithi049 ceph-mon[31942]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:27 smithi129 ceph-mon[36561]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:27 smithi154 ceph-mon[36471]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T08:11:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:28.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T08:11:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:28.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T08:11:28.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:29 smithi154 ceph-mon[36471]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:29 smithi154 ceph-mon[36471]: Deploying daemon osd.5 on smithi154 2024-03-16T08:11:29.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:29 smithi129 ceph-mon[36561]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:29.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:29 smithi129 ceph-mon[36561]: Deploying daemon osd.5 on smithi154 2024-03-16T08:11:29.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:29 smithi049 ceph-mon[31942]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:29.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:29 smithi049 ceph-mon[31942]: Deploying daemon osd.5 on smithi154 2024-03-16T08:11:31.419 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:31 smithi154 ceph-mon[36471]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:31.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:31 smithi129 ceph-mon[36561]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:31.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:31 smithi049 ceph-mon[31942]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:33.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:33 smithi154 ceph-mon[36471]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:33.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:33 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:33.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:33 smithi049 ceph-mon[31942]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:33.589 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.589 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.589 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:33 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:33 smithi129 ceph-mon[36561]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:33 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:35.380 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:35 smithi154 ceph-mon[36471]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:35.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:35 smithi129 ceph-mon[36561]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:35.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:35 smithi049 ceph-mon[31942]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:36.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.630+0000 7f1b2b7fe700 1 -- 172.21.15.154:0/2674559966 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1b34062710 con 0x7f1b200a2910 2024-03-16T08:11:36.630 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 5 on host 'smithi154' 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 msgr2=0x7f1b200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 0x7f1b200a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009cb0 tx=0x7f1b24009260 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 msgr2=0x7f1b34198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c9a0 tx=0x7f1b3000cd60 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 shutdown_connections 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b200a2910 0x7f1b200a4dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b34102da0 0x7f1b3419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b34101200 0x7f1b34198450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 --2- 172.21.15.154:0/2674559966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b34102400 0x7f1b34198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 >> 172.21.15.154:0/2674559966 conn(0x7f1b340fc7c0 msgr2=0x7f1b341056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 shutdown_connections 2024-03-16T08:11:36.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:11:36.632+0000 7f1b3c5f0700 1 -- 172.21.15.154:0/2674559966 wait complete. 2024-03-16T08:11:37.319 DEBUG:teuthology.orchestra.run.smithi154:osd.5> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.5.service 2024-03-16T08:11:37.321 INFO:tasks.cephadm:Deploying osd.6 on smithi049 with /dev/vg_nvme/lv_2... 2024-03-16T08:11:37.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T08:11:37.573 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:37.573 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:37.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:37 smithi154 ceph-mon[36471]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:37.583 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:37 smithi129 ceph-mon[36561]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:37.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:37.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:37.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:37.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:37 smithi049 ceph-mon[31942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:38 smithi049 ceph-mon[31942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:39.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T08:11:39.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T08:11:39.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:39.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-mon[36471]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.226 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:11:38.974+0000 7f1ebd9b0700 -1 osd.5 0 waiting for initial osdmap 2024-03-16T08:11:39.226 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:11:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:11:38.980+0000 7f1eb97a3700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:11:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T08:11:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T08:11:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: from='osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:38 smithi129 ceph-mon[36561]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:11:39.783 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852461 s, 1.2 GB/s 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 2024-03-16T08:11:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-16T08:11:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:39 smithi049 ceph-mon[31942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:11:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:39 smithi049 ceph-mon[31942]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T08:11:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:39 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.322 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:39 smithi154 ceph-mon[36471]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:11:40.323 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:39 smithi154 ceph-mon[36471]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T08:11:40.323 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.323 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:39 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:39 smithi129 ceph-mon[36561]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:11:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:39 smithi129 ceph-mon[36561]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T08:11:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:39 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:40.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-03-16T08:11:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:11:41.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] boot 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi154 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:41.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:40 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:11:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:11:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] boot 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi154 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:40 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: osd.5 [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] boot 2024-03-16T08:11:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi154 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:11:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:40 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:42.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:41 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 8540M 2024-03-16T08:11:42.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:41 smithi049 ceph-mon[31942]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T08:11:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:41 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 8540M 2024-03-16T08:11:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:41 smithi129 ceph-mon[36561]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T08:11:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 -- 172.21.15.49:0/2885752707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 msgr2=0x7f45e010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 --2- 172.21.15.49:0/2885752707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e010a500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45d800b0a0 tx=0x7f45d800b3b0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 -- 172.21.15.49:0/2885752707 shutdown_connections 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 --2- 172.21.15.49:0/2885752707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e010a500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 --2- 172.21.15.49:0/2885752707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 0x7f45e0105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.385+0000 7f45e7123700 1 --2- 172.21.15.49:0/2885752707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- 172.21.15.49:0/2885752707 >> 172.21.15.49:0/2885752707 conn(0x7f45e00fc860 msgr2=0x7f45e00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- 172.21.15.49:0/2885752707 shutdown_connections 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- 172.21.15.49:0/2885752707 wait complete. 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 Processor -- start 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- start start 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 0x7f45e01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45e010dfe0 con 0x7f45e0108090 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45e010de60 con 0x7f45e0105760 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.386+0000 7f45e7123700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45e010e160 con 0x7f45e0102e30 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45480/0 (socket says 172.21.15.49:45480) 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 -- 172.21.15.49:0/3465704647 learned_addr learned my addr 172.21.15.49:0/3465704647 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e4ebf700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 0x7f45e01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45dffff700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 -- 172.21.15.49:0/3465704647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 msgr2=0x7f45e01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 0x7f45e01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 -- 172.21.15.49:0/3465704647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 msgr2=0x7f45e0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:11:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 -- 172.21.15.49:0/3465704647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e01a0200 con 0x7f45e0108090 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45dffff700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e56c0700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d010 tx=0x7f45d8004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8021070 con 0x7f45e0108090 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e01a0490 con 0x7f45e0108090 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8007ca0 con 0x7f45e0108090 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80055d0 con 0x7f45e0108090 2024-03-16T08:11:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.387+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01a0a10 con 0x7f45e0108090 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.389+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e0068f90 con 0x7f45e0108090 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.390+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f45d8024070 con 0x7f45e0108090 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.390+0000 7f45ddffb700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 0x7f45d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.390+0000 7f45e4ebf700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 0x7f45d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.390+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f45d800f020 con 0x7f45e0108090 2024-03-16T08:11:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.391+0000 7f45e4ebf700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 0x7f45d00a4d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006fd0 tx=0x7f45d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:11:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.395+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f45d802b020 con 0x7f45e0108090 2024-03-16T08:11:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:41 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 8540M 2024-03-16T08:11:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:41 smithi154 ceph-mon[36471]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T08:11:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:11:42.611+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f45e0062710 con 0x7f45d00a2890 2024-03-16T08:11:43.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:42 smithi049 ceph-mon[31942]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:43.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:42 smithi049 ceph-mon[31942]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T08:11:43.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:42 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:43.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:42 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:43.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:42 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:42 smithi129 ceph-mon[36561]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:42 smithi129 ceph-mon[36561]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T08:11:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:42 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:42 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:42 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:43.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:42 smithi154 ceph-mon[36471]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:43.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:42 smithi154 ceph-mon[36471]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T08:11:43.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:42 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:11:43.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:42 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:11:43.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:42 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:44.108 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:44 smithi049 ceph-mon[31942]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:44.108 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:44 smithi049 ceph-mon[31942]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T08:11:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:44 smithi129 ceph-mon[36561]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:44 smithi129 ceph-mon[36561]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T08:11:44.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:44 smithi154 ceph-mon[36471]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:11:44.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:44 smithi154 ceph-mon[36471]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T08:11:45.261 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:45 smithi129 ceph-mon[36561]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:45.261 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:45 smithi129 ceph-mon[36561]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T08:11:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:45 smithi049 ceph-mon[31942]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:45 smithi049 ceph-mon[31942]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T08:11:45.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:45 smithi154 ceph-mon[36471]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:45.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:45 smithi154 ceph-mon[36471]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T08:11:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:46 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]: dispatch 2024-03-16T08:11:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:46 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]': finished 2024-03-16T08:11:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:46 smithi049 ceph-mon[31942]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T08:11:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:11:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:46 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2143669045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:46.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:46 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]: dispatch 2024-03-16T08:11:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:46 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]': finished 2024-03-16T08:11:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:46 smithi129 ceph-mon[36561]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T08:11:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:11:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:46 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2143669045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:46 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]: dispatch 2024-03-16T08:11:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:46 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3911472340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36672b95-4a0b-4ffc-9103-5f9e2c187b9b"}]': finished 2024-03-16T08:11:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:46 smithi154 ceph-mon[36471]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T08:11:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:11:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:46 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2143669045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:11:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:47 smithi129 ceph-mon[36561]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:47 smithi154 ceph-mon[36471]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:47.786 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:47 smithi049 ceph-mon[31942]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:11:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:49 smithi129 ceph-mon[36561]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-16T08:11:49.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:49 smithi049 ceph-mon[31942]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-16T08:11:49.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:49 smithi154 ceph-mon[36471]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-16T08:11:51.534 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:51 smithi049 ceph-mon[31942]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T08:11:51.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:51 smithi129 ceph-mon[36561]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T08:11:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:51 smithi154 ceph-mon[36471]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T08:11:53.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:53 smithi049 ceph-mon[31942]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:53.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T08:11:53.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:53.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:53 smithi129 ceph-mon[36561]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:53.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T08:11:53.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:53.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:53 smithi154 ceph-mon[36471]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:53.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T08:11:53.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:11:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:54 smithi154 ceph-mon[36471]: Deploying daemon osd.6 on smithi049 2024-03-16T08:11:54.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:54 smithi049 ceph-mon[31942]: Deploying daemon osd.6 on smithi049 2024-03-16T08:11:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:54 smithi129 ceph-mon[36561]: Deploying daemon osd.6 on smithi049 2024-03-16T08:11:55.432 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:55 smithi049 ceph-mon[31942]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:55 smithi129 ceph-mon[36561]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:55 smithi154 ceph-mon[36471]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:57.434 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:57 smithi049 ceph-mon[31942]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:57 smithi129 ceph-mon[36561]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:57 smithi154 ceph-mon[36471]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:59 smithi129 ceph-mon[36561]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:11:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:59 smithi154 ceph-mon[36471]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:11:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:11:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:59 smithi049 ceph-mon[31942]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:11:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:11:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:11:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:01 smithi129 ceph-mon[36561]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:01.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:01 smithi154 ceph-mon[36471]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:01.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:01 smithi049 ceph-mon[31942]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:01.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.833+0000 7f45ddffb700 1 -- 172.21.15.49:0/3465704647 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45e0062710 con 0x7f45d00a2890 2024-03-16T08:12:01.833 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 6 on host 'smithi049' 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 msgr2=0x7f45d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 0x7f45d00a4d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006fd0 tx=0x7f45d4008040 comp rx=0 tx=0).stop 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 msgr2=0x7f45e019fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d010 tx=0x7f45d8004ef0 comp rx=0 tx=0).stop 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 shutdown_connections 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f45d00a2890 0x7f45d00a4d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e0108090 0x7f45e019fb60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45e0102e30 0x7f45e01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 --2- 172.21.15.49:0/3465704647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e0105760 0x7f45e0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 >> 172.21.15.49:0/3465704647 conn(0x7f45e00fc860 msgr2=0x7f45e0103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 shutdown_connections 2024-03-16T08:12:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:01.835+0000 7f45e7123700 1 -- 172.21.15.49:0/3465704647 wait complete. 2024-03-16T08:12:02.525 DEBUG:teuthology.orchestra.run.smithi049:osd.6> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.6.service 2024-03-16T08:12:02.528 INFO:tasks.cephadm:Deploying osd.7 on smithi129 with /dev/vg_nvme/lv_2... 2024-03-16T08:12:02.528 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T08:12:02.773 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:03.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-mon[31942]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T08:12:03.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-mon[31942]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T08:12:03.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-mon[31942]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:12:03.819 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:03.819 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:12:03.809+0000 7fdffadec700 -1 osd.6 0 waiting for initial osdmap 2024-03-16T08:12:03.819 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:12:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:12:03.817+0000 7fdff6bdf700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:12:04.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:03 smithi129 ceph-mon[36561]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T08:12:04.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:03 smithi129 ceph-mon[36561]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T08:12:04.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:03 smithi129 ceph-mon[36561]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:12:04.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:03 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:03 smithi154 ceph-mon[36471]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T08:12:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:03 smithi154 ceph-mon[36471]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T08:12:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:03 smithi154 ceph-mon[36471]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:12:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:03 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112805 s, 930 MB/s 2024-03-16T08:12:04.904 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-03-16T08:12:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:04 smithi049 ceph-mon[31942]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:04 smithi049 ceph-mon[31942]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:12:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:04 smithi049 ceph-mon[31942]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T08:12:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:04 smithi129 ceph-mon[36561]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:05.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:04 smithi129 ceph-mon[36561]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:12:05.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:04 smithi129 ceph-mon[36561]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T08:12:05.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:04 smithi154 ceph-mon[36471]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T08:12:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:04 smithi154 ceph-mon[36471]: from='osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:12:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:04 smithi154 ceph-mon[36471]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T08:12:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:05.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-03-16T08:12:05.739 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:12:06.071 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:05 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:12:06.071 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:05 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:12:06.071 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:05 smithi129 ceph-mon[36561]: osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] boot 2024-03-16T08:12:06.071 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:05 smithi129 ceph-mon[36561]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T08:12:06.071 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:05 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:12:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:05 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:12:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:05 smithi154 ceph-mon[36471]: osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] boot 2024-03-16T08:12:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:05 smithi154 ceph-mon[36471]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T08:12:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:05 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:12:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:05 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:12:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:05 smithi049 ceph-mon[31942]: osd.6 [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] boot 2024-03-16T08:12:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:05 smithi049 ceph-mon[31942]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T08:12:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi049 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:06 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi049 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:06 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi049 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:07.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:06 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 -- 172.21.15.129:0/1359476710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 msgr2=0x7fb1801009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 --2- 172.21.15.129:0/1359476710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb1801009e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb170009990 tx=0x7fb170009ca0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 -- 172.21.15.129:0/1359476710 shutdown_connections 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 --2- 172.21.15.129:0/1359476710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb18010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 --2- 172.21.15.129:0/1359476710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb1801009e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 --2- 172.21.15.129:0/1359476710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 0x7fb180103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 -- 172.21.15.129:0/1359476710 >> 172.21.15.129:0/1359476710 conn(0x7fb1800fbf40 msgr2=0x7fb1800fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 -- 172.21.15.129:0/1359476710 shutdown_connections 2024-03-16T08:12:07.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.312+0000 7fb187e7c700 1 -- 172.21.15.129:0/1359476710 wait complete. 2024-03-16T08:12:07.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 Processor -- start 2024-03-16T08:12:07.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 -- start start 2024-03-16T08:12:07.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:07.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 0x7fb180101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:07.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb180101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb18010dfb0 con 0x7fb1801035a0 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb18010de30 con 0x7fb1801005c0 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb187e7c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb18010e130 con 0x7fb180103f40 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54596/0 (socket says 172.21.15.129:54596) 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 -- 172.21.15.129:0/4219769102 learned_addr learned my addr 172.21.15.129:0/4219769102 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb186419700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb180101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185417700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 0x7fb180101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 -- 172.21.15.129:0/4219769102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 msgr2=0x7fb180101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb180101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 -- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 msgr2=0x7fb180101550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 0x7fb180101550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:07.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb185c18700 1 -- 172.21.15.129:0/4219769102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb180101fd0 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.313+0000 7fb186419700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb180101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb185c18700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb170000c00 tx=0x7fb170009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170027070 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb170022470 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb180102260 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17000f6c0 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.314+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1801a70c0 con 0x7fb1801005c0 2024-03-16T08:12:07.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.315+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb180068f90 con 0x7fb1801005c0 2024-03-16T08:12:07.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.317+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb17002a070 con 0x7fb1801005c0 2024-03-16T08:12:07.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.317+0000 7fb176ffd700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 0x7fb16c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:07.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.317+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fb1700cf1c0 con 0x7fb1801005c0 2024-03-16T08:12:07.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.318+0000 7fb185417700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 0x7fb16c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:07.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.318+0000 7fb185417700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 0x7fb16c0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb17c006fd0 tx=0x7fb17c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:07.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.321+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb170013070 con 0x7fb1801005c0 2024-03-16T08:12:07.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:07.523+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb1800627d0 con 0x7fb16c0a2890 2024-03-16T08:12:08.097 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:08.097 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:08.097 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:08.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:08.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:08.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:09.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:08 smithi129 ceph-mon[36561]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:09.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:08 smithi129 ceph-mon[36561]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:08 smithi154 ceph-mon[36471]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:09.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:08 smithi154 ceph-mon[36471]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:08 smithi049 ceph-mon[31942]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:08 smithi049 ceph-mon[31942]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/702027109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]': finished 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T08:12:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/702027109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]': finished 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T08:12:11.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/702027109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]: dispatch 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313db83e-0f1f-4748-b054-77708bd6a27e"}]': finished 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T08:12:11.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:12.109 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:11 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/2940604665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:12.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:11 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/2940604665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:12.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:11 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/2940604665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:13.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:12 smithi129 ceph-mon[36561]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:13.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:12 smithi154 ceph-mon[36471]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:12 smithi049 ceph-mon[31942]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:15.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:14 smithi049 ceph-mon[31942]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:15.106 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:14 smithi129 ceph-mon[36561]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:15.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:14 smithi154 ceph-mon[36471]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:17.089 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:16 smithi129 ceph-mon[36561]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:17.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:16 smithi154 ceph-mon[36471]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:17.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:16 smithi049 ceph-mon[31942]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T08:12:18.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:17 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:18.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T08:12:18.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:17 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:18.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T08:12:18.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:17 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:19.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:18 smithi129 ceph-mon[36561]: Deploying daemon osd.7 on smithi129 2024-03-16T08:12:19.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:18 smithi129 ceph-mon[36561]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:19.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:18 smithi154 ceph-mon[36471]: Deploying daemon osd.7 on smithi129 2024-03-16T08:12:19.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:18 smithi154 ceph-mon[36471]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:18 smithi049 ceph-mon[31942]: Deploying daemon osd.7 on smithi129 2024-03-16T08:12:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:18 smithi049 ceph-mon[31942]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:21.056 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:20 smithi129 ceph-mon[36561]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:21.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:20 smithi154 ceph-mon[36471]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:21.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:20 smithi049 ceph-mon[31942]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:23.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:22 smithi129 ceph-mon[36561]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:23.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:22 smithi154 ceph-mon[36471]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:23.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:22 smithi049 ceph-mon[31942]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:24.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:24.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:24.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:24.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:24.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:24.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:25.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:24 smithi129 ceph-mon[36561]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:25.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:24 smithi154 ceph-mon[36471]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:25.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:24 smithi049 ceph-mon[31942]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:26.792 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 7 on host 'smithi129' 2024-03-16T08:12:26.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.791+0000 7fb176ffd700 1 -- 172.21.15.129:0/4219769102 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1800627d0 con 0x7fb16c0a2890 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 msgr2=0x7fb16c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 0x7fb16c0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb17c006fd0 tx=0x7fb17c008040 comp rx=0 tx=0).stop 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 msgr2=0x7fb180102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb170000c00 tx=0x7fb170009f90 comp rx=0 tx=0).stop 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 shutdown_connections 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb16c0a2890 0x7fb16c0a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb180103f40 0x7fb180101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1801005c0 0x7fb180102f00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:26.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 --2- 172.21.15.129:0/4219769102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1801035a0 0x7fb180101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:26.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.793+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 >> 172.21.15.129:0/4219769102 conn(0x7fb1800fbf40 msgr2=0x7fb180106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:26.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.794+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 shutdown_connections 2024-03-16T08:12:26.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:12:26.794+0000 7fb187e7c700 1 -- 172.21.15.129:0/4219769102 wait complete. 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:27.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:27 smithi129 ceph-mon[36561]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:27.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:27.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:27 smithi049 ceph-mon[31942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.439 DEBUG:teuthology.orchestra.run.smithi129:osd.7> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.7.service 2024-03-16T08:12:27.441 INFO:tasks.cephadm:Deploying osd.8 on smithi154 with /dev/vg_nvme/lv_2... 2024-03-16T08:12:27.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T08:12:27.471 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:27.471 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.471 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.471 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.472 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:27 smithi154 ceph-mon[36471]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T08:12:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:29.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-mon[36561]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.000 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:12:28.756+0000 7fae7d0f7700 -1 osd.7 0 waiting for initial osdmap 2024-03-16T08:12:29.001 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:12:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:12:28.766+0000 7fae78eea700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:29.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:28 smithi154 ceph-mon[36471]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: from='osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:28 smithi049 ceph-mon[31942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111944 s, 937 MB/s 2024-03-16T08:12:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] boot 2024-03-16T08:12:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T08:12:30.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:12:30.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T08:12:30.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.068 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.068 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] boot 2024-03-16T08:12:30.068 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T08:12:30.068 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: osd.7 [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] boot 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T08:12:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:12:30.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-03-16T08:12:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:30 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:30 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:12:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:12:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T08:12:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T08:12:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:30 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:32.187 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:31 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi129 2024-03-16T08:12:32.187 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:31 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 5693M 2024-03-16T08:12:32.187 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:31 smithi154 ceph-mon[36471]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T08:12:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:31 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi129 2024-03-16T08:12:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:31 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 5693M 2024-03-16T08:12:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:31 smithi049 ceph-mon[31942]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T08:12:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:31 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi129 2024-03-16T08:12:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:31 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 5693M 2024-03-16T08:12:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:31 smithi129 ceph-mon[36561]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T08:12:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.399+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/1587827679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8105690 msgr2=0x7fd9e8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.399+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/1587827679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8105690 0x7fd9e8107a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009990 tx=0x7fd9dc009ca0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/1587827679 shutdown_connections 2024-03-16T08:12:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/1587827679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8107fc0 0x7fd9e810a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/1587827679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8102d60 0x7fd9e8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/1587827679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8105690 0x7fd9e8107a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/1587827679 >> 172.21.15.154:0/1587827679 conn(0x7fd9e80fc790 msgr2=0x7fd9e80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/1587827679 shutdown_connections 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/1587827679 wait complete. 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 Processor -- start 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.400+0000 7fd9ee7dc700 1 -- start start 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 0x7fd9e81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8105690 0x7fd9e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e810dee0 con 0x7fd9e8105690 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e810dd60 con 0x7fd9e8102d60 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e810e060 con 0x7fd9e8107fc0 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 0x7fd9e81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57734/0 (socket says 172.21.15.154:57734) 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 -- 172.21.15.154:0/3650151499 learned_addr learned my addr 172.21.15.154:0/3650151499 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e77fe700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8105690 0x7fd9e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 -- 172.21.15.154:0/3650151499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 msgr2=0x7fd9e81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 0x7fd9e81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 -- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8105690 msgr2=0x7fd9e81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8105690 0x7fd9e81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 -- 172.21.15.154:0/3650151499 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e81a0110 con 0x7fd9e8107fc0 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e7fff700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 0x7fd9e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:12:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ecd79700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000efc0 tx=0x7fd9e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e00092e0 con 0x7fd9e8107fc0 2024-03-16T08:12:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e0004bb0 con 0x7fd9e8107fc0 2024-03-16T08:12:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e81a03a0 con 0x7fd9e8107fc0 2024-03-16T08:12:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0004280 con 0x7fd9e8107fc0 2024-03-16T08:12:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.401+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e81a0890 con 0x7fd9e8107fc0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.402+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e8068f90 con 0x7fd9e8107fc0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.403+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd9e001b030 con 0x7fd9e8107fc0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.404+0000 7fd9e57fa700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 0x7fd9d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.404+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7fd9e00cd9a0 con 0x7fd9e8107fc0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.404+0000 7fd9e7fff700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 0x7fd9d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.405+0000 7fd9e7fff700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 0x7fd9d00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8010420 tx=0x7fd9d80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.409+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd9e0022020 con 0x7fd9e8107fc0 2024-03-16T08:12:32.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:32.613+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd9e8062710 con 0x7fd9d00a2890 2024-03-16T08:12:33.188 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: pgmap v178: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:33.188 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-16T08:12:33.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T08:12:33.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:33.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:33.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:33.221 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: pgmap v178: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:33.222 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-16T08:12:33.222 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T08:12:33.222 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:33.222 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:33.222 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:32 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: pgmap v178: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:32 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:34.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:33 smithi049 ceph-mon[31942]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:34.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:33 smithi049 ceph-mon[31942]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T08:12:34.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:33 smithi154 ceph-mon[36471]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:34.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:33 smithi154 ceph-mon[36471]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T08:12:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:33 smithi129 ceph-mon[36561]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:33 smithi129 ceph-mon[36561]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T08:12:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:34 smithi154 ceph-mon[36471]: pgmap v181: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:34 smithi049 ceph-mon[31942]: pgmap v181: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:35.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:34 smithi129 ceph-mon[36561]: pgmap v181: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-03-16T08:12:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:35 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/2435020448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:35 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:35 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]': finished 2024-03-16T08:12:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:35 smithi154 ceph-mon[36471]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T08:12:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:35 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/2435020448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:35 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:35 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]': finished 2024-03-16T08:12:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:35 smithi049 ceph-mon[31942]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T08:12:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:35 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/2435020448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:35 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]: dispatch 2024-03-16T08:12:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:35 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c82b22c1-18d5-40dd-8a86-428405308e1a"}]': finished 2024-03-16T08:12:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:35 smithi129 ceph-mon[36561]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T08:12:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:36 smithi154 ceph-mon[36471]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:36 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/131267854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:36 smithi154 ceph-mon[36471]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-16T08:12:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:36 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:12:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:36 smithi049 ceph-mon[31942]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:36 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/131267854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:36 smithi049 ceph-mon[31942]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-16T08:12:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:36 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:12:37.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:36 smithi129 ceph-mon[36561]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:37.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:36 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/131267854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:12:37.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:36 smithi129 ceph-mon[36561]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-16T08:12:37.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:36 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:12:39.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:38 smithi154 ceph-mon[36471]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:39.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:38 smithi049 ceph-mon[31942]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:38 smithi129 ceph-mon[36561]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:41.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:40 smithi154 ceph-mon[36471]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:41.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:40 smithi049 ceph-mon[31942]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:40 smithi129 ceph-mon[36561]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:42.979 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:42 smithi154 ceph-mon[36471]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:42.980 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:42 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T08:12:42.980 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:42 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:43.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:42 smithi049 ceph-mon[31942]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:43.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:42 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T08:12:43.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:42 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:42 smithi129 ceph-mon[36561]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:42 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T08:12:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:42 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:44.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:43 smithi154 ceph-mon[36471]: Deploying daemon osd.8 on smithi154 2024-03-16T08:12:44.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:43 smithi049 ceph-mon[31942]: Deploying daemon osd.8 on smithi154 2024-03-16T08:12:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:43 smithi129 ceph-mon[36561]: Deploying daemon osd.8 on smithi154 2024-03-16T08:12:44.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:44 smithi154 ceph-mon[36471]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:45.261 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:44 smithi129 ceph-mon[36561]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:44 smithi049 ceph-mon[31942]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:47.170 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:46 smithi154 ceph-mon[36471]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:47.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:46 smithi049 ceph-mon[31942]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:47.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:46 smithi129 ceph-mon[36561]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:49.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:48 smithi154 ceph-mon[36471]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:49.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.224 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:49.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:48 smithi049 ceph-mon[31942]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:49.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:48 smithi129 ceph-mon[36561]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:51.216 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:50 smithi154 ceph-mon[36471]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:51.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:50 smithi049 ceph-mon[31942]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:50 smithi129 ceph-mon[36561]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:52.131 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 8 on host 'smithi154' 2024-03-16T08:12:52.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.130+0000 7fd9e57fa700 1 -- 172.21.15.154:0/3650151499 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd9e8062710 con 0x7fd9d00a2890 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 msgr2=0x7fd9d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 0x7fd9d00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8010420 tx=0x7fd9d80073d0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 msgr2=0x7fd9e819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000efc0 tx=0x7fd9e000c3a0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 shutdown_connections 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd9d00a2890 0x7fd9d00a4d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9e8107fc0 0x7fd9e819fa70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9e8102d60 0x7fd9e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 --2- 172.21.15.154:0/3650151499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e8105690 0x7fd9e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:52.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 >> 172.21.15.154:0/3650151499 conn(0x7fd9e80fc790 msgr2=0x7fd9e81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:52.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 shutdown_connections 2024-03-16T08:12:52.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:12:52.133+0000 7fd9ee7dc700 1 -- 172.21.15.154:0/3650151499 wait complete. 2024-03-16T08:12:52.800 DEBUG:teuthology.orchestra.run.smithi154:osd.8> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.8.service 2024-03-16T08:12:52.802 INFO:tasks.cephadm:Deploying osd.9 on smithi049 with /dev/vg_nvme/lv_1... 2024-03-16T08:12:52.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T08:12:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.193 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:54 smithi049 ceph-mon[31942]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T08:12:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:54 smithi049 ceph-mon[31942]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:54 smithi049 ceph-mon[31942]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T08:12:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:54 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:54 smithi049 ceph-mon[31942]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:54 smithi129 ceph-mon[36561]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T08:12:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:54 smithi129 ceph-mon[36561]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:54 smithi129 ceph-mon[36561]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T08:12:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:54 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:54 smithi129 ceph-mon[36561]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-mon[36471]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T08:12:54.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-mon[36471]: from='osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-mon[36471]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T08:12:54.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:54.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-mon[36471]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:12:54.674 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:12:54.537+0000 7f7d45b09700 -1 osd.8 0 waiting for initial osdmap 2024-03-16T08:12:54.674 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:12:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:12:54.546+0000 7f7d3d8f4700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:12:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T08:12:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T08:12:55.287 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-16T08:12:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-16T08:12:55.288 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834679 s, 1.3 GB/s 2024-03-16T08:12:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:55 smithi049 ceph-mon[31942]: from='osd.8 ' entity='osd.8' 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:55 smithi129 ceph-mon[36561]: from='osd.8 ' entity='osd.8' 2024-03-16T08:12:55.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:55.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:12:55.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T08:12:55.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:55.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:55 smithi154 ceph-mon[36471]: from='osd.8 ' entity='osd.8' 2024-03-16T08:12:55.960 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-03-16T08:12:56.216 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] boot 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T08:12:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:56 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] boot 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T08:12:56.352 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:56 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: osd.8 [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] boot 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T08:12:56.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:56 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:12:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T08:12:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.425 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.426 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.426 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:57.426 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.426 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.426 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T08:12:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T08:12:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:12:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 -- 172.21.15.49:0/3784058756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103ec0 msgr2=0x7f421010a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 --2- 172.21.15.49:0/3784058756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103ec0 0x7f421010a770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f420800b0d0 tx=0x7f420800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 -- 172.21.15.49:0/3784058756 shutdown_connections 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 --2- 172.21.15.49:0/3784058756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103ec0 0x7f421010a770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 --2- 172.21.15.49:0/3784058756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210102320 0x7f4210102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 --2- 172.21.15.49:0/3784058756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103520 0x7f4210103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.942+0000 7f42152e9700 1 -- 172.21.15.49:0/3784058756 >> 172.21.15.49:0/3784058756 conn(0x7f42100fd8e0 msgr2=0x7f42100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- 172.21.15.49:0/3784058756 shutdown_connections 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- 172.21.15.49:0/3784058756 wait complete. 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 Processor -- start 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- start start 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210103ec0 0x7f4210074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f421010def0 con 0x7f4210103520 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f421010dd70 con 0x7f4210103ec0 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.943+0000 7f42152e9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f421010e070 con 0x7f4210102320 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:40396/0 (socket says 172.21.15.49:40396) 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59780/0 (socket says 172.21.15.49:59780) 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 -- 172.21.15.49:0/2669492621 learned_addr learned my addr 172.21.15.49:0/2669492621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420f7fe700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210103ec0 0x7f4210074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 -- 172.21.15.49:0/2669492621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210103ec0 msgr2=0x7f4210074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210103ec0 0x7f4210074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 -- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 msgr2=0x7f42100746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 -- 172.21.15.49:0/2669492621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100754b0 con 0x7f4210102320 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420e7fc700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f420effd700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f420400d980 tx=0x7f420400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.944+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204016070 con 0x7f4210102320 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.945+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101a25d0 con 0x7f4210102320 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.945+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420400be30 con 0x7f4210102320 2024-03-16T08:12:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.945+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400f3c0 con 0x7f4210102320 2024-03-16T08:12:57.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.945+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42101a2ae0 con 0x7f4210102320 2024-03-16T08:12:57.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.945+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4210068f90 con 0x7f4210102320 2024-03-16T08:12:57.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.947+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4204019070 con 0x7f4210102320 2024-03-16T08:12:57.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.947+0000 7f41f7fff700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 0x7f41f80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:12:57.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.948+0000 7f420e7fc700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 0x7f41f80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:12:57.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.948+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f42040ce5d0 con 0x7f4210102320 2024-03-16T08:12:57.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.948+0000 7f420e7fc700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 0x7f41f80a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4200005950 tx=0x7f420000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:12:57.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:57.951+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4204020030 con 0x7f4210102320 2024-03-16T08:12:58.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:12:58.166+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4210062710 con 0x7f41f80a2890 2024-03-16T08:12:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:58 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi154 2024-03-16T08:12:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:58 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 5693M 2024-03-16T08:12:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:58 smithi154 ceph-mon[36471]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T08:12:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:58 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi154 2024-03-16T08:12:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:58 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 5693M 2024-03-16T08:12:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:58 smithi049 ceph-mon[31942]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T08:12:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:58 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi154 2024-03-16T08:12:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:58 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 5693M 2024-03-16T08:12:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:58 smithi129 ceph-mon[36561]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T08:12:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:12:59.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T08:12:59.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:59.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:59.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:59.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:12:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:12:59.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T08:12:59.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:59.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:59.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:59.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:12:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:12:59.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:12:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:00 smithi154 ceph-mon[36471]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T08:13:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:00 smithi049 ceph-mon[31942]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T08:13:00.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:00 smithi129 ceph-mon[36561]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T08:13:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:01 smithi154 ceph-mon[36471]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]: dispatch 2024-03-16T08:13:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]': finished 2024-03-16T08:13:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:01 smithi154 ceph-mon[36471]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T08:13:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:01 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:01 smithi049 ceph-mon[31942]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]: dispatch 2024-03-16T08:13:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]': finished 2024-03-16T08:13:01.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:01 smithi049 ceph-mon[31942]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T08:13:01.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:01 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:01 smithi129 ceph-mon[36561]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]: dispatch 2024-03-16T08:13:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4103286490' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "693d0e23-6cdd-41f2-b7c4-f9bb3686edbf"}]': finished 2024-03-16T08:13:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:01 smithi129 ceph-mon[36561]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T08:13:01.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:01 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3804737331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3804737331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:02.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:02 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3804737331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:03.438 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:03 smithi049 ceph-mon[31942]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-16T08:13:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:03 smithi154 ceph-mon[36471]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-16T08:13:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:03 smithi129 ceph-mon[36561]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-16T08:13:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:05 smithi049 ceph-mon[31942]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:13:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:05 smithi154 ceph-mon[36471]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:13:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:05 smithi129 ceph-mon[36561]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:13:07.437 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:07 smithi049 ceph-mon[31942]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-16T08:13:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:07 smithi154 ceph-mon[36471]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-16T08:13:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:07 smithi129 ceph-mon[36561]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-16T08:13:09.466 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:09 smithi049 ceph-mon[31942]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-16T08:13:09.466 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:09 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T08:13:09.466 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:09 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:09.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:09 smithi154 ceph-mon[36471]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-16T08:13:09.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:09 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T08:13:09.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:09 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:09.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:09 smithi129 ceph-mon[36561]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-16T08:13:09.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:09 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T08:13:09.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:09 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:10 smithi154 ceph-mon[36471]: Deploying daemon osd.9 on smithi049 2024-03-16T08:13:10.531 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:10 smithi049 ceph-mon[31942]: Deploying daemon osd.9 on smithi049 2024-03-16T08:13:10.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:10 smithi129 ceph-mon[36561]: Deploying daemon osd.9 on smithi049 2024-03-16T08:13:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:11 smithi049 ceph-mon[31942]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:13:11.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:11 smithi154 ceph-mon[36471]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:13:11.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:11 smithi129 ceph-mon[36561]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T08:13:13.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:13 smithi154 ceph-mon[36471]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:13.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:13 smithi049 ceph-mon[31942]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:13.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:13 smithi129 ceph-mon[36561]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:15 smithi049 ceph-mon[31942]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:15 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:15.411 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:15 smithi129 ceph-mon[36561]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:15.411 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.411 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.411 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:15 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:15.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:15 smithi154 ceph-mon[36471]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:15.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:15.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:15 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:17 smithi154 ceph-mon[36471]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:17 smithi049 ceph-mon[31942]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:17 smithi129 ceph-mon[36561]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:18.477 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 9 on host 'smithi049' 2024-03-16T08:13:18.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.477+0000 7f41f7fff700 1 -- 172.21.15.49:0/2669492621 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4210062710 con 0x7f41f80a2890 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 msgr2=0x7f41f80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 0x7f41f80a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4200005950 tx=0x7f420000b410 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 msgr2=0x7f4210076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f420400d980 tx=0x7f420400dc90 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 shutdown_connections 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41f80a2890 0x7f41f80a4d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4210103ec0 0x7f4210074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210102320 0x7f4210076070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 --2- 172.21.15.49:0/2669492621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4210103520 0x7f42100746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 >> 172.21.15.49:0/2669492621 conn(0x7f42100fd8e0 msgr2=0x7f42101067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:13:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 shutdown_connections 2024-03-16T08:13:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:13:18.480+0000 7f42152e9700 1 -- 172.21.15.49:0/2669492621 wait complete. 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T08:13:18.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T08:13:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:19.092 DEBUG:teuthology.orchestra.run.smithi049:osd.9> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.9.service 2024-03-16T08:13:19.094 INFO:tasks.cephadm:Deploying osd.10 on smithi129 with /dev/vg_nvme/lv_1... 2024-03-16T08:13:19.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T08:13:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:13:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-mon[31942]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T08:13:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-mon[31942]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T08:13:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-mon[31942]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:13:20.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:20.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:13:20.168+0000 7f15ce306700 -1 osd.9 0 waiting for initial osdmap 2024-03-16T08:13:20.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:13:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:13:20.183+0000 7f15ca0f9700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:13:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:20 smithi154 ceph-mon[36471]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T08:13:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:20 smithi154 ceph-mon[36471]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T08:13:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:20 smithi154 ceph-mon[36471]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:13:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:20 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:20.498 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:20 smithi129 ceph-mon[36561]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T08:13:20.498 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:20 smithi129 ceph-mon[36561]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T08:13:20.498 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:20 smithi129 ceph-mon[36561]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-16T08:13:20.498 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:20 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:21 smithi129 ceph-mon[36561]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:21.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:21 smithi129 ceph-mon[36561]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:13:21.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:21 smithi129 ceph-mon[36561]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T08:13:21.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:21 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.343 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:21 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113252 s, 926 MB/s 2024-03-16T08:13:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-03-16T08:13:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:21 smithi154 ceph-mon[36471]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:21 smithi154 ceph-mon[36471]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:13:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:21 smithi154 ceph-mon[36471]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T08:13:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:21 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:21 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:21 smithi049 ceph-mon[31942]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:21 smithi049 ceph-mon[31942]: from='osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-16T08:13:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:21 smithi049 ceph-mon[31942]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T08:13:21.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:21 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:21.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:21 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-03-16T08:13:22.397 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] boot 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T08:13:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:22 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] boot 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T08:13:22.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:22 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: osd.9 [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] boot 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T08:13:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:22 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi049 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.406 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:23.407 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:23.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi049 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T08:13:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T08:13:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi049 2024-03-16T08:13:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:23.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- 172.21.15.129:0/4191702453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b81057c0 msgr2=0x7fd2b8107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:23.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 --2- 172.21.15.129:0/4191702453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b81057c0 0x7fd2b8107bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009990 tx=0x7fd2a4009ca0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- 172.21.15.129:0/4191702453 shutdown_connections 2024-03-16T08:13:23.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 --2- 172.21.15.129:0/4191702453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81080f0 0x7fd2b810a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 --2- 172.21.15.129:0/4191702453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b8102e90 0x7fd2b8105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 --2- 172.21.15.129:0/4191702453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b81057c0 0x7fd2b8107bb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- 172.21.15.129:0/4191702453 >> 172.21.15.129:0/4191702453 conn(0x7fd2b80fc860 msgr2=0x7fd2b80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- 172.21.15.129:0/4191702453 shutdown_connections 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- 172.21.15.129:0/4191702453 wait complete. 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 Processor -- start 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.912+0000 7fd2bec66700 1 -- start start 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81057c0 0x7fd2b8100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b81080f0 0x7fd2b8101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810df80 con 0x7fd2b81057c0 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810de00 con 0x7fd2b8102e90 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b810e100 con 0x7fd2b81080f0 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49594/0 (socket says 172.21.15.129:49594) 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 -- 172.21.15.129:0/2599950494 learned_addr learned my addr 172.21.15.129:0/2599950494 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bd203700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b81080f0 0x7fd2b8101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2b7fff700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81057c0 0x7fd2b8100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 -- 172.21.15.129:0/2599950494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b81080f0 msgr2=0x7fd2b8101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b81080f0 0x7fd2b8101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 -- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81057c0 msgr2=0x7fd2b8100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81057c0 0x7fd2b8100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 -- 172.21.15.129:0/2599950494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b8101c30 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bca02700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c9b0 tx=0x7fd2ac00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac007b00 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b81a2670 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac007ca0 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.913+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac018730 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.914+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b81a2c30 con 0x7fd2b8102e90 2024-03-16T08:13:23.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.915+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b8068f90 con 0x7fd2b8102e90 2024-03-16T08:13:23.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.916+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd2ac0188d0 con 0x7fd2b8102e90 2024-03-16T08:13:23.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.916+0000 7fd2b5ffb700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 0x7fd2a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:23.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.916+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd2ac0ce230 con 0x7fd2b8102e90 2024-03-16T08:13:23.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.916+0000 7fd2b7fff700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 0x7fd2a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:23.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.917+0000 7fd2b7fff700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 0x7fd2a80a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b5c0 tx=0x7fd2a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:13:23.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:23.921+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd2ac014030 con 0x7fd2b8102e90 2024-03-16T08:13:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:24.124+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd2b8062710 con 0x7fd2a80a2890 2024-03-16T08:13:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:24 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:24 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:24 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:25 smithi129 ceph-mon[36561]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:25 smithi129 ceph-mon[36561]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:25.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:25 smithi154 ceph-mon[36471]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:25.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:25 smithi154 ceph-mon[36471]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:25 smithi049 ceph-mon[31942]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:25 smithi049 ceph-mon[31942]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/1193909725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]': finished 2024-03-16T08:13:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T08:13:27.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:27 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/1193909725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]': finished 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T08:13:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:27 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/1193909725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]: dispatch 2024-03-16T08:13:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fdf3874-86b2-4175-a20c-41ac7241c806"}]': finished 2024-03-16T08:13:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T08:13:27.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:27 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:28 smithi154 ceph-mon[36471]: from='client.? 172.21.15.129:0/3337906745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:28 smithi049 ceph-mon[31942]: from='client.? 172.21.15.129:0/3337906745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:28 smithi129 ceph-mon[36561]: from='client.? 172.21.15.129:0/3337906745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:29.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:29 smithi154 ceph-mon[36471]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:29.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:29 smithi049 ceph-mon[31942]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:29.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:29 smithi129 ceph-mon[36561]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:31.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:31 smithi129 ceph-mon[36561]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:31.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:31 smithi154 ceph-mon[36471]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:31.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:31 smithi049 ceph-mon[31942]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:33.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:33 smithi129 ceph-mon[36561]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:33.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:33 smithi154 ceph-mon[36471]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:33.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:33 smithi049 ceph-mon[31942]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:35.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:35 smithi129 ceph-mon[36561]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:35.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T08:13:35.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:35.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:35 smithi154 ceph-mon[36471]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:35.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T08:13:35.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:35 smithi049 ceph-mon[31942]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T08:13:35.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:36 smithi129 ceph-mon[36561]: Deploying daemon osd.10 on smithi129 2024-03-16T08:13:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:36 smithi154 ceph-mon[36471]: Deploying daemon osd.10 on smithi129 2024-03-16T08:13:36.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:36 smithi049 ceph-mon[31942]: Deploying daemon osd.10 on smithi129 2024-03-16T08:13:37.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:37 smithi154 ceph-mon[36471]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:37.490 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:37 smithi129 ceph-mon[36561]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:37.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:37 smithi049 ceph-mon[31942]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:39.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:39 smithi154 ceph-mon[36471]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:39.525 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:39 smithi129 ceph-mon[36561]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:39.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:39 smithi049 ceph-mon[31942]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:41 smithi129 ceph-mon[36561]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:41 smithi154 ceph-mon[36471]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:41 smithi049 ceph-mon[31942]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:43.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:43 smithi154 ceph-mon[36471]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:43 smithi049 ceph-mon[31942]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:43 smithi129 ceph-mon[36561]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:44.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.096+0000 7fd2b5ffb700 1 -- 172.21.15.129:0/2599950494 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd2b8062710 con 0x7fd2a80a2890 2024-03-16T08:13:44.097 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 10 on host 'smithi129' 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 msgr2=0x7fd2a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 0x7fd2a80a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b5c0 tx=0x7fd2a4005fb0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 msgr2=0x7fd2b81027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c9b0 tx=0x7fd2ac00ccc0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 shutdown_connections 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd2a80a2890 0x7fd2a80a4d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2b81080f0 0x7fd2b8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2b8102e90 0x7fd2b81027f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 --2- 172.21.15.129:0/2599950494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b81057c0 0x7fd2b8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 >> 172.21.15.129:0/2599950494 conn(0x7fd2b80fc860 msgr2=0x7fd2b81037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 shutdown_connections 2024-03-16T08:13:44.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-03-16T08:13:44.098+0000 7fd2bec66700 1 -- 172.21.15.129:0/2599950494 wait complete. 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.662 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T08:13:44.662 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.662 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:44.662 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:13:44.803 DEBUG:teuthology.orchestra.run.smithi129:osd.10> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.10.service 2024-03-16T08:13:44.806 INFO:tasks.cephadm:Deploying osd.11 on smithi154 with /dev/vg_nvme/lv_1... 2024-03-16T08:13:44.806 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T08:13:45.040 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:13:45.416 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:45.416 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T08:13:45.416 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T08:13:45.416 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:45.416 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:45.417 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:45 smithi154 ceph-mon[36471]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-mon[36561]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:45.611 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:13:45.344+0000 7f4ba0ae8700 -1 osd.10 0 waiting for initial osdmap 2024-03-16T08:13:45.611 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:13:45 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:13:45.355+0000 7f4b9c8db700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:13:45.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:45.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T08:13:45.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T08:13:45.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:45.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: from='osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:45.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:45 smithi049 ceph-mon[31942]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-03-16T08:13:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:46 smithi154 ceph-mon[36471]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:13:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:46 smithi154 ceph-mon[36471]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T08:13:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:46.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:46 smithi129 ceph-mon[36561]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:13:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:46 smithi129 ceph-mon[36561]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T08:13:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:46 smithi049 ceph-mon[31942]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-03-16T08:13:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:46 smithi049 ceph-mon[31942]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T08:13:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112823 s, 929 MB/s 2024-03-16T08:13:47.196 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-03-16T08:13:47.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:13:47.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:13:47.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:47.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] boot 2024-03-16T08:13:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T08:13:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] boot 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T08:13:47.537 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:13:47.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:13:47.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T08:13:47.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: osd.10 [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] boot 2024-03-16T08:13:47.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T08:13:47.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:13:47.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-03-16T08:13:48.217 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:13:48.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T08:13:48.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:48.595 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:48 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:48.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:48.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:48 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T08:13:48.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:13:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:48 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:13:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:49 smithi129 ceph-mon[36561]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:49 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi129 2024-03-16T08:13:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:49 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 4270M 2024-03-16T08:13:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:49 smithi129 ceph-mon[36561]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T08:13:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:49 smithi129 ceph-mon[36561]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T08:13:49.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:49 smithi154 ceph-mon[36471]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:49.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:49 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi129 2024-03-16T08:13:49.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:49 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 4270M 2024-03-16T08:13:49.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:49 smithi154 ceph-mon[36471]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T08:13:49.617 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:49 smithi154 ceph-mon[36471]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T08:13:49.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:49 smithi049 ceph-mon[31942]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:49.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:49 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi129 2024-03-16T08:13:49.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:49 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 4270M 2024-03-16T08:13:49.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:49 smithi049 ceph-mon[31942]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T08:13:49.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:49 smithi049 ceph-mon[31942]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T08:13:49.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- 172.21.15.154:0/4189536795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c1035c0 msgr2=0x7f181c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:49.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 --2- 172.21.15.154:0/4189536795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c1035c0 0x7f181c103a20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1810009990 tx=0x7f1810009ca0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- 172.21.15.154:0/4189536795 shutdown_connections 2024-03-16T08:13:49.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 --2- 172.21.15.154:0/4189536795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c103f60 0x7f181c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 --2- 172.21.15.154:0/4189536795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1005e0 0x7f181c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 --2- 172.21.15.154:0/4189536795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c1035c0 0x7f181c103a20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- 172.21.15.154:0/4189536795 >> 172.21.15.154:0/4189536795 conn(0x7f181c0fbf60 msgr2=0x7f181c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- 172.21.15.154:0/4189536795 shutdown_connections 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- 172.21.15.154:0/4189536795 wait complete. 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 Processor -- start 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.973+0000 7f1822505700 1 -- start start 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 0x7f181c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1035c0 0x7f181c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10df00 con 0x7f181c1005e0 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10dd80 con 0x7f181c1035c0 2024-03-16T08:13:49.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10e080 con 0x7f181c103f60 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52244/0 (socket says 172.21.15.154:52244) 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 -- 172.21.15.154:0/2828826469 learned_addr learned my addr 172.21.15.154:0/2828826469 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f181b7fe700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1035c0 0x7f181c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f181bfff700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 0x7f181c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 -- 172.21.15.154:0/2828826469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1035c0 msgr2=0x7f181c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1035c0 0x7f181c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 -- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 msgr2=0x7f181c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 0x7f181c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 -- 172.21.15.154:0/2828826469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1a0110 con 0x7f181c103f60 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f181bfff700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 0x7f181c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1820aa2700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1814007d40 tx=0x7f181400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:13:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814016020 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1a0400 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f181400ad60 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.974+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814011600 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.975+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1a08f0 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.976+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c068f90 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.976+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1814019020 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.977+0000 7f18197fa700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 0x7f18040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.977+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7f18140ce120 con 0x7f181c103f60 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.977+0000 7f181bfff700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 0x7f18040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.978+0000 7f181bfff700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 0x7f18040a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f180c005950 tx=0x7f180c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:13:49.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:49.982+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1814020030 con 0x7f181c103f60 2024-03-16T08:13:50.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:13:50.185+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f181c062710 con 0x7f18040a2890 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:51.048 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:51.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:51.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T08:13:51.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:51.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:51.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:51.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:13:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:13:53.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:52 smithi154 ceph-mon[36471]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-16T08:13:53.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:52 smithi049 ceph-mon[31942]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-16T08:13:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:52 smithi129 ceph-mon[36561]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/2790087308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]': finished 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:13:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:53 smithi154 ceph-mon[36471]: from='client.? 172.21.15.154:0/3804036334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/2790087308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]': finished 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:13:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:53 smithi049 ceph-mon[31942]: from='client.? 172.21.15.154:0/3804036334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/2790087308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]: dispatch 2024-03-16T08:13:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9663681-0e97-4339-931e-27dc3f68bb58"}]': finished 2024-03-16T08:13:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T08:13:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:13:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:53 smithi129 ceph-mon[36561]: from='client.? 172.21.15.154:0/3804036334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:13:55.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:54 smithi154 ceph-mon[36471]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-16T08:13:55.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:54 smithi049 ceph-mon[31942]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-16T08:13:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:54 smithi129 ceph-mon[36561]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-16T08:13:57.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:56 smithi154 ceph-mon[36471]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T08:13:57.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:56 smithi049 ceph-mon[31942]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T08:13:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:56 smithi129 ceph-mon[36561]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T08:13:59.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:13:58 smithi154 ceph-mon[36471]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T08:13:59.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:13:58 smithi049 ceph-mon[31942]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T08:13:59.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:13:58 smithi129 ceph-mon[36561]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T08:14:01.257 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:00 smithi154 ceph-mon[36471]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:01.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T08:14:01.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:01.258 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:00 smithi154 ceph-mon[36471]: Deploying daemon osd.11 on smithi154 2024-03-16T08:14:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:00 smithi049 ceph-mon[31942]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T08:14:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:00 smithi049 ceph-mon[31942]: Deploying daemon osd.11 on smithi154 2024-03-16T08:14:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:00 smithi129 ceph-mon[36561]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T08:14:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:00 smithi129 ceph-mon[36561]: Deploying daemon osd.11 on smithi154 2024-03-16T08:14:03.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:02 smithi049 ceph-mon[31942]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:02 smithi129 ceph-mon[36561]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:03.413 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:02 smithi154 ceph-mon[36471]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:05.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:04 smithi154 ceph-mon[36471]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:05.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:04 smithi049 ceph-mon[31942]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:05.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:04 smithi129 ceph-mon[36561]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:07 smithi154 ceph-mon[36471]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:07 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:07 smithi049 ceph-mon[31942]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:07 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:07 smithi129 ceph-mon[36561]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:07 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:09.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:09 smithi154 ceph-mon[36471]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:09.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:09 smithi049 ceph-mon[31942]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:09.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:09 smithi129 ceph-mon[36561]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:10.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.411+0000 7f18197fa700 1 -- 172.21.15.154:0/2828826469 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f181c062710 con 0x7f18040a2890 2024-03-16T08:14:10.413 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 11 on host 'smithi154' 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 msgr2=0x7f18040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 0x7f18040a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f180c005950 tx=0x7f180c0058e0 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 msgr2=0x7f181c19fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1814007d40 tx=0x7f181400a340 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 shutdown_connections 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f18040a2890 0x7f18040a4d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f181c103f60 0x7f181c19fa70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f181c1005e0 0x7f181c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 --2- 172.21.15.154:0/2828826469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f181c1035c0 0x7f181c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 >> 172.21.15.154:0/2828826469 conn(0x7f181c0fbf60 msgr2=0x7f181c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 shutdown_connections 2024-03-16T08:14:10.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:10.414+0000 7f1822505700 1 -- 172.21.15.154:0/2828826469 wait complete. 2024-03-16T08:14:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:10 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:10.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:10 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:10 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:11.129 DEBUG:teuthology.orchestra.run.smithi154:osd.11> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.11.service 2024-03-16T08:14:11.131 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-16T08:14:11.131 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T08:14:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:14:11.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-mon[36471]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T08:14:11.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-mon[36471]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T08:14:11.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:11.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-mon[36471]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-mon[36471]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:11 smithi049 ceph-mon[31942]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T08:14:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:11 smithi049 ceph-mon[31942]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T08:14:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:11 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:11 smithi049 ceph-mon[31942]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:11 smithi049 ceph-mon[31942]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:11 smithi129 ceph-mon[36561]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T08:14:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:11 smithi129 ceph-mon[36561]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T08:14:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:11 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:11 smithi129 ceph-mon[36561]: from='osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:11 smithi129 ceph-mon[36561]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-03-16T08:14:11.888 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:14:11.874+0000 7f244b38b700 -1 osd.11 0 waiting for initial osdmap 2024-03-16T08:14:11.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:14:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:14:11.887+0000 7f2442975700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:14:12.673 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:14:12.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T08:14:12.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:12.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:12 smithi154 ceph-mon[36471]: from='osd.11 ' entity='osd.11' 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:12 smithi049 ceph-mon[31942]: from='osd.11 ' entity='osd.11' 2024-03-16T08:14:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-03-16T08:14:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T08:14:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T08:14:12.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:12 smithi129 ceph-mon[36561]: from='osd.11 ' entity='osd.11' 2024-03-16T08:14:13.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- 172.21.15.49:0/4072601893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 msgr2=0x7fdb04102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:13.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- 172.21.15.49:0/4072601893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb04102710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b0d0 tx=0x7fdafc00b3e0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- 172.21.15.49:0/4072601893 shutdown_connections 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- 172.21.15.49:0/4072601893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb04103e90 0x7fdb0410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- 172.21.15.49:0/4072601893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb04102710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- 172.21.15.49:0/4072601893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb041034f0 0x7fdb04103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- 172.21.15.49:0/4072601893 >> 172.21.15.49:0/4072601893 conn(0x7fdb040fd8b0 msgr2=0x7fdb040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- 172.21.15.49:0/4072601893 shutdown_connections 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- 172.21.15.49:0/4072601893 wait complete. 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 Processor -- start 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- start start 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb041034f0 0x7fdb04198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:13.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb04103e90 0x7fdb0419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410df30 con 0x7fdb041022f0 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410ddb0 con 0x7fdb041034f0 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.210+0000 7fdb0be44700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410e0b0 con 0x7fdb04103e90 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48210/0 (socket says 172.21.15.49:48210) 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 -- 172.21.15.49:0/3194434886 learned_addr learned my addr 172.21.15.49:0/3194434886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb093df700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb041034f0 0x7fdb04198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb0a3e1700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb04103e90 0x7fdb0419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 -- 172.21.15.49:0/3194434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb04103e90 msgr2=0x7fdb0419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb04103e90 0x7fdb0419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 -- 172.21.15.49:0/3194434886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb041034f0 msgr2=0x7fdb04198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb041034f0 0x7fdb04198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 -- 172.21.15.49:0/3194434886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb041a0230 con 0x7fdb041022f0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb09be0700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f80 tx=0x7fdafc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc021020 con 0x7fdb041022f0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb041a04c0 con 0x7fdb041022f0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdafc01d950 con 0x7fdb041022f0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb041a0ad0 con 0x7fdb041022f0 2024-03-16T08:14:13.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.211+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc005450 con 0x7fdb041022f0 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.212+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf0005360 con 0x7fdb041022f0 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.214+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdafc024020 con 0x7fdb041022f0 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.214+0000 7fdb02ffd700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.214+0000 7fdb093df700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.215+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdafc00f020 con 0x7fdb041022f0 2024-03-16T08:14:13.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.215+0000 7fdb093df700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 0x7fdaec0a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009a10 tx=0x7fdaf8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:13.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.218+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdafc09d0b0 con 0x7fdb041022f0 2024-03-16T08:14:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.423+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdaf0005920 con 0x7fdb041022f0 2024-03-16T08:14:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.423+0000 7fdb02ffd700 1 -- 172.21.15.49:0/3194434886 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fdafc025e40 con 0x7fdb041022f0 2024-03-16T08:14:13.425 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 msgr2=0x7fdaec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 0x7fdaec0a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009a10 tx=0x7fdaf8009190 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 msgr2=0x7fdb041985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f80 tx=0x7fdafc007d90 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 shutdown_connections 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb04103e90 0x7fdb0419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb041022f0 0x7fdb041985d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 --2- 172.21.15.49:0/3194434886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb041034f0 0x7fdb04198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.426+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 >> 172.21.15.49:0/3194434886 conn(0x7fdb040fd8b0 msgr2=0x7fdb04106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.427+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 shutdown_connections 2024-03-16T08:14:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:13.427+0000 7fdb0be44700 1 -- 172.21.15.49:0/3194434886 wait complete. 2024-03-16T08:14:13.517 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: purged_snaps scrub starts 2024-03-16T08:14:13.517 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: purged_snaps scrub ok 2024-03-16T08:14:13.517 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.517 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] boot 2024-03-16T08:14:13.518 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T08:14:13.518 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.518 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:13 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3194434886' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: purged_snaps scrub starts 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: purged_snaps scrub ok 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] boot 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T08:14:13.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.724 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:13 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3194434886' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:14:13.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: purged_snaps scrub starts 2024-03-16T08:14:13.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: purged_snaps scrub ok 2024-03-16T08:14:13.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: osd.11 [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] boot 2024-03-16T08:14:13.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T08:14:13.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:14:13.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:13 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3194434886' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:14:14.280 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1710576852,"num_in_osds":12,"osd_in_since":1710576832,"num_remapped_pgs":0} 2024-03-16T08:14:14.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:14:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:14:14.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:14.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:14 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.981 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:14.981 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:14.982 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:14 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:15.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:14 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:16.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:15 smithi049 ceph-mon[31942]: Detected new or changed devices on smithi154 2024-03-16T08:14:16.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:15 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 4270M 2024-03-16T08:14:16.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:15 smithi154 ceph-mon[36471]: Detected new or changed devices on smithi154 2024-03-16T08:14:16.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:15 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 4270M 2024-03-16T08:14:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:15 smithi129 ceph-mon[36561]: Detected new or changed devices on smithi154 2024-03-16T08:14:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:15 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 4270M 2024-03-16T08:14:16.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 -- 172.21.15.49:0/3251672797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2410ede0 msgr2=0x7f9c241111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:16.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/3251672797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2410ede0 0x7f9c241111d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b0a0 tx=0x7f9c1c00b3b0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 -- 172.21.15.49:0/3251672797 shutdown_connections 2024-03-16T08:14:16.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/3251672797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2410ede0 0x7f9c241111d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/3251672797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2406a9d0 0x7f9c2406adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/3251672797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 0x7f9c2410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 -- 172.21.15.49:0/3251672797 >> 172.21.15.49:0/3251672797 conn(0x7f9c24077220 msgr2=0x7f9c24077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:16.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 -- 172.21.15.49:0/3251672797 shutdown_connections 2024-03-16T08:14:16.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.406+0000 7f9c2a63e700 1 -- 172.21.15.49:0/3251672797 wait complete. 2024-03-16T08:14:16.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 Processor -- start 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 -- start start 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 0x7f9c241a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2410ede0 0x7f9c2419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c24114850 con 0x7f9c2406a9d0 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241146d0 con 0x7f9c2406b2f0 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c2a63e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241149d0 con 0x7f9c2410ede0 2024-03-16T08:14:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48228/0 (socket says 172.21.15.49:48228) 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 -- 172.21.15.49:0/1921831183 learned_addr learned my addr 172.21.15.49:0/1921831183 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c237fe700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 0x7f9c241a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c28bdb700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2410ede0 0x7f9c2419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 -- 172.21.15.49:0/1921831183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2410ede0 msgr2=0x7f9c2419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2410ede0 0x7f9c2419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 -- 172.21.15.49:0/1921831183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 msgr2=0x7f9c241a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 0x7f9c241a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.407+0000 7f9c23fff700 1 -- 172.21.15.49:0/1921831183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2419b1d0 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c23fff700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b530 tx=0x7f9c1400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14015020 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2419b4c0 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c241a7780 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c14011d10 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.408+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400f2e0 con 0x7f9c2406a9d0 2024-03-16T08:14:16.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.409+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c24068f90 con 0x7f9c2406a9d0 2024-03-16T08:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.410+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9c14018020 con 0x7f9c2406a9d0 2024-03-16T08:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.411+0000 7f9c217fa700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 0x7f9c100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.411+0000 7f9c237fe700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 0x7f9c100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.411+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9c140ce280 con 0x7f9c2406a9d0 2024-03-16T08:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.411+0000 7f9c237fe700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 0x7f9c100a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c24100dd0 tx=0x7f9c0c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:16.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.414+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c1401f020 con 0x7f9c2406a9d0 2024-03-16T08:14:16.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.618+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c241a7a70 con 0x7f9c2406a9d0 2024-03-16T08:14:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.619+0000 7f9c217fa700 1 -- 172.21.15.49:0/1921831183 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18421 (secure 0 0 0) 0x7f9c1409b7c0 con 0x7f9c2406a9d0 2024-03-16T08:14:16.620 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:16.620 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:14:13.878528+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 msgr2=0x7f9c100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 0x7f9c100a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c24100dd0 tx=0x7f9c0c0073d0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 msgr2=0x7f9c241a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b530 tx=0x7f9c1400b840 comp rx=0 tx=0).stop 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 shutdown_connections 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9c100a2890 0x7f9c100a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c2410ede0 0x7f9c2419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c2406a9d0 0x7f9c241a08b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 --2- 172.21.15.49:0/1921831183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c2406b2f0 0x7f9c241a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 >> 172.21.15.49:0/1921831183 conn(0x7f9c24077220 msgr2=0x7f9c241101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 shutdown_connections 2024-03-16T08:14:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:16.621+0000 7f9c2a63e700 1 -- 172.21.15.49:0/1921831183 wait complete. 2024-03-16T08:14:17.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:16 smithi049 ceph-mon[31942]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:17.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:16 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1921831183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:17.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:16 smithi154 ceph-mon[36471]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:17.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:16 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1921831183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:16 smithi129 ceph-mon[36561]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:16 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1921831183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:17.406 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:14:17.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-16T08:14:17.663 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:14:19.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:18 smithi049 ceph-mon[31942]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:19.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:18 smithi154 ceph-mon[36471]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:18 smithi129 ceph-mon[36561]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 -- 172.21.15.49:0/648795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0107660 msgr2=0x7f39e010f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 --2- 172.21.15.49:0/648795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0107660 0x7f39e010f1e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b0d0 tx=0x7f39d800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 -- 172.21.15.49:0/648795604 shutdown_connections 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 --2- 172.21.15.49:0/648795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0107660 0x7f39e010f1e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 --2- 172.21.15.49:0/648795604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e01025b0 0x7f39e0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 --2- 172.21.15.49:0/648795604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e0106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 -- 172.21.15.49:0/648795604 >> 172.21.15.49:0/648795604 conn(0x7f39e00793f0 msgr2=0x7f39e0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 -- 172.21.15.49:0/648795604 shutdown_connections 2024-03-16T08:14:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.410+0000 7f39e51ee700 1 -- 172.21.15.49:0/648795604 wait complete. 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 Processor -- start 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 -- start start 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0107660 0x7f39e0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0112620 con 0x7f39e01025b0 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01124a0 con 0x7f39e0107660 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39e51ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01127a0 con 0x7f39e0102f60 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39ded9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:19.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39ded9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48244/0 (socket says 172.21.15.49:48244) 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39ded9d700 1 -- 172.21.15.49:0/1762368284 learned_addr learned my addr 172.21.15.49:0/1762368284 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39de59c700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.411+0000 7f39df59e700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0107660 0x7f39e0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 -- 172.21.15.49:0/1762368284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 msgr2=0x7f39e019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 -- 172.21.15.49:0/1762368284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0107660 msgr2=0x7f39e0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0107660 0x7f39e0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 -- 172.21.15.49:0/1762368284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e0199000 con 0x7f39e01025b0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39de59c700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39ded9d700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39d400d950 tx=0x7f39d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4016070 con 0x7f39e01025b0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01992f0 con 0x7f39e01025b0 2024-03-16T08:14:19.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d400fd70 con 0x7f39e01025b0 2024-03-16T08:14:19.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a5500 con 0x7f39e01025b0 2024-03-16T08:14:19.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.412+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400e610 con 0x7f39e01025b0 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.413+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0103a10 con 0x7f39e01025b0 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.415+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f39d4019070 con 0x7f39e01025b0 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.415+0000 7f39c7fff700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 0x7f39c80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.415+0000 7f39de59c700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 0x7f39c80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.415+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39d40ce3f0 con 0x7f39e01025b0 2024-03-16T08:14:19.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.416+0000 7f39de59c700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 0x7f39c80a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39e0108a70 tx=0x7f39d000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:19.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.419+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f39d409b930 con 0x7f39e01025b0 2024-03-16T08:14:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.621+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39e0199d50 con 0x7f39e01025b0 2024-03-16T08:14:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.621+0000 7f39c7fff700 1 -- 172.21.15.49:0/1762368284 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f39d409b230 con 0x7f39e01025b0 2024-03-16T08:14:19.623 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:14:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.623+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 msgr2=0x7f39c80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.623+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 0x7f39c80a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39e0108a70 tx=0x7f39d000b3c0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.623+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 msgr2=0x7f39e019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.623+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39d400d950 tx=0x7f39d400dc60 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.623+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 shutdown_connections 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f39c80a2890 0x7f39c80a4d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0107660 0x7f39e0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01025b0 0x7f39e019e6a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 --2- 172.21.15.49:0/1762368284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39e0102f60 0x7f39e019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 >> 172.21.15.49:0/1762368284 conn(0x7f39e00793f0 msgr2=0x7f39e0100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 shutdown_connections 2024-03-16T08:14:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:19.624+0000 7f39e51ee700 1 -- 172.21.15.49:0/1762368284 wait complete. 2024-03-16T08:14:20.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:19 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1762368284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:20.282 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:19 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1762368284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.a on smithi049 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.d on smithi049 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.g on smithi049 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.j on smithi049 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.b on smithi129 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.e on smithi129 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.h on smithi129 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.k on smithi129 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.c on smithi154 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.f on smithi154 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.i on smithi154 2024-03-16T08:14:20.310 INFO:tasks.cephadm:Adding mds.l on smithi154 2024-03-16T08:14:20.311 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l' 2024-03-16T08:14:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:19 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1762368284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:20.563 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:14:20.943 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:20 smithi154 ceph-mon[36471]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:21.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:20 smithi049 ceph-mon[31942]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:21.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:20 smithi129 ceph-mon[36561]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- 172.21.15.154:0/1385258804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103ea0 msgr2=0x7fd18c10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- 172.21.15.154:0/1385258804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103ea0 0x7fd18c10a750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b0a0 tx=0x7fd18400b3b0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- 172.21.15.154:0/1385258804 shutdown_connections 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- 172.21.15.154:0/1385258804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103ea0 0x7fd18c10a750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- 172.21.15.154:0/1385258804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c102300 0x7fd18c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- 172.21.15.154:0/1385258804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c103500 0x7fd18c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- 172.21.15.154:0/1385258804 >> 172.21.15.154:0/1385258804 conn(0x7fd18c0fd8e0 msgr2=0x7fd18c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- 172.21.15.154:0/1385258804 shutdown_connections 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- 172.21.15.154:0/1385258804 wait complete. 2024-03-16T08:14:22.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 Processor -- start 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- start start 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 0x7fd18c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 0x7fd18c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10dee0 con 0x7fd18c103ea0 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10dd60 con 0x7fd18c102300 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.221+0000 7fd1940a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10e060 con 0x7fd18c103500 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41654/0 (socket says 172.21.15.154:41654) 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 -- 172.21.15.154:0/672292217 learned_addr learned my addr 172.21.15.154:0/672292217 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191e42700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 0x7fd18c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd192643700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 0x7fd18c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:22.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 -- 172.21.15.154:0/672292217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 msgr2=0x7fd18c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 0x7fd18c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 -- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 msgr2=0x7fd18c19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 0x7fd18c19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 -- 172.21.15.154:0/672292217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c1a0100 con 0x7fd18c103500 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd192643700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 0x7fd18c19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191e42700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 0x7fd18c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd191641700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00d9b0 tx=0x7fd17c00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c017070 con 0x7fd18c103500 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c004d10 con 0x7fd18c103500 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005500 con 0x7fd18c103500 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1a03f0 con 0x7fd18c103500 2024-03-16T08:14:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.222+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a0930 con 0x7fd18c103500 2024-03-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.223+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c068f90 con 0x7fd18c103500 2024-03-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.224+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd17c004750 con 0x7fd18c103500 2024-03-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.225+0000 7fd182ffd700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 0x7fd1780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.225+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd17c0ce8d0 con 0x7fd18c103500 2024-03-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.225+0000 7fd191e42700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 0x7fd1780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:22.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.226+0000 7fd191e42700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 0x7fd1780a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd18c103360 tx=0x7fd188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:22.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.229+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd17c020020 con 0x7fd18c103500 2024-03-16T08:14:22.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.434+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}) -- 0x7fd18c0628d0 con 0x7fd1780a2910 2024-03-16T08:14:22.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.442+0000 7fd182ffd700 1 -- 172.21.15.154:0/672292217 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd18c0628d0 con 0x7fd1780a2910 2024-03-16T08:14:22.442 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mds.all update... 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 msgr2=0x7fd1780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 0x7fd1780a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd18c103360 tx=0x7fd188009260 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 msgr2=0x7fd18c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00d9b0 tx=0x7fd17c00dd70 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 shutdown_connections 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd1780a2910 0x7fd1780a4dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18c103ea0 0x7fd18c19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18c102300 0x7fd18c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 --2- 172.21.15.154:0/672292217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd18c103500 0x7fd18c1989e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 >> 172.21.15.154:0/672292217 conn(0x7fd18c0fd8e0 msgr2=0x7fd18c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 shutdown_connections 2024-03-16T08:14:22.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-16T08:14:22.444+0000 7fd1940a6700 1 -- 172.21.15.154:0/672292217 wait complete. 2024-03-16T08:14:23.198 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:22 smithi154 ceph-mon[36471]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:23.199 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:22 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:23.199 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:22 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:23.229 DEBUG:teuthology.orchestra.run.smithi049:mds.a> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.a.service 2024-03-16T08:14:23.232 DEBUG:teuthology.orchestra.run.smithi049:mds.d> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.d.service 2024-03-16T08:14:23.263 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:22 smithi049 ceph-mon[31942]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:23.263 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:22 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:23.263 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:22 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:23.264 DEBUG:teuthology.orchestra.run.smithi049:mds.g> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.g.service 2024-03-16T08:14:23.308 DEBUG:teuthology.orchestra.run.smithi049:mds.j> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.j.service 2024-03-16T08:14:23.350 DEBUG:teuthology.orchestra.run.smithi129:mds.b> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.b.service 2024-03-16T08:14:23.353 DEBUG:teuthology.orchestra.run.smithi129:mds.e> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.e.service 2024-03-16T08:14:23.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:22 smithi129 ceph-mon[36561]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:23.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:22 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:23.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:22 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:23.361 DEBUG:teuthology.orchestra.run.smithi129:mds.h> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.h.service 2024-03-16T08:14:23.403 DEBUG:teuthology.orchestra.run.smithi129:mds.k> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.k.service 2024-03-16T08:14:23.447 DEBUG:teuthology.orchestra.run.smithi154:mds.c> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.c.service 2024-03-16T08:14:23.450 DEBUG:teuthology.orchestra.run.smithi154:mds.f> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.f.service 2024-03-16T08:14:23.492 DEBUG:teuthology.orchestra.run.smithi154:mds.i> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.i.service 2024-03-16T08:14:23.535 DEBUG:teuthology.orchestra.run.smithi154:mds.l> sudo journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.l.service 2024-03-16T08:14:23.578 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-16T08:14:23.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:14:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 -- 172.21.15.49:0/1403533257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 msgr2=0x7fc0501046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1403533257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc0501046f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc040009900 tx=0x7fc04002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 -- 172.21.15.49:0/1403533257 shutdown_connections 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1403533257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 0x7fc0501099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1403533257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 0x7fc050107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1403533257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc0501046f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 -- 172.21.15.49:0/1403533257 >> 172.21.15.49:0/1403533257 conn(0x7fc0500fbd10 msgr2=0x7fc0500fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.698+0000 7fc0564e8640 1 -- 172.21.15.49:0/1403533257 shutdown_connections 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 -- 172.21.15.49:0/1403533257 wait complete. 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 Processor -- start 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 -- start start 2024-03-16T08:14:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 0x7fc0501980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 0x7fc05019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc05010d560 con 0x7fc050102300 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc05010d3e0 con 0x7fc050107560 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.699+0000 7fc0564e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc05010d6e0 con 0x7fc050104c30 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38156/0 (socket says 172.21.15.49:38156) 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 -- 172.21.15.49:0/1144285686 learned_addr learned my addr 172.21.15.49:0/1144285686 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04f7fe640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 0x7fc0501980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc054a85640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 0x7fc05019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 -- 172.21.15.49:0/1144285686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 msgr2=0x7fc0501980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 0x7fc0501980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 -- 172.21.15.49:0/1144285686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 msgr2=0x7fc05019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 0x7fc05019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 -- 172.21.15.49:0/1144285686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05019f7d0 con 0x7fc050102300 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04ffff640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc040005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040043020 con 0x7fc050102300 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05019fa60 con 0x7fc050102300 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0400054e0 con 0x7fc050102300 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040003e60 con 0x7fc050102300 2024-03-16T08:14:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.700+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05019ffb0 con 0x7fc050102300 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.701+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0500fd800 con 0x7fc050102300 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.702+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc040046020 con 0x7fc050102300 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.703+0000 7fc04d7fa640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 0x7fc0380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.703+0000 7fc04f7fe640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 0x7fc0380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.703+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc0400f0930 con 0x7fc050102300 2024-03-16T08:14:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.703+0000 7fc04f7fe640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 0x7fc0380a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0501990c0 tx=0x7fc044008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:23.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.707+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc04004d020 con 0x7fc050102300 2024-03-16T08:14:23.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.892+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0500623e0 con 0x7fc050102300 2024-03-16T08:14:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.893+0000 7fc04d7fa640 1 -- 172.21.15.49:0/1144285686 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18421 (secure 0 0 0) 0x7fc04003c350 con 0x7fc050102300 2024-03-16T08:14:23.894 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:23.894 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:14:13.878528+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.894+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 msgr2=0x7fc0380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.894+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 0x7fc0380a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0501990c0 tx=0x7fc044008040 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.894+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 msgr2=0x7fc050197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.894+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc040005dd0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 shutdown_connections 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc050107560 0x7fc05019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc050104c30 0x7fc0501980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc0380a2800 0x7fc0380a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 --2- 172.21.15.49:0/1144285686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc050102300 0x7fc050197bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 >> 172.21.15.49:0/1144285686 conn(0x7fc0500fbd10 msgr2=0x7fc050102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 shutdown_connections 2024-03-16T08:14:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:23.895+0000 7fc0564e8640 1 -- 172.21.15.49:0/1144285686 wait complete. 2024-03-16T08:14:23.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:14:23.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.018+0000 7f23100b2640 1 -- 172.21.15.49:0/809843792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 msgr2=0x7f2308101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.018+0000 7f23100b2640 1 --2- 172.21.15.49:0/809843792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f2308101d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.018+0000 7f23100b2640 1 -- 172.21.15.49:0/809843792 shutdown_connections 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.018+0000 7f23100b2640 1 --2- 172.21.15.49:0/809843792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 --2- 172.21.15.49:0/809843792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f2308101d50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 --2- 172.21.15.49:0/809843792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 0x7f2308100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- 172.21.15.49:0/809843792 >> 172.21.15.49:0/809843792 conn(0x7f23080fbc70 msgr2=0x7f23080fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- 172.21.15.49:0/809843792 shutdown_connections 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- 172.21.15.49:0/809843792 wait complete. 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 Processor -- start 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- start start 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 0x7f230810b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d480 con 0x7f23081018d0 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d300 con 0x7f23081006e0 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f23100b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d600 con 0x7f2308102290 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f230d64d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f230d64d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38176/0 (socket says 172.21.15.49:38176) 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.019+0000 7f230d64d640 1 -- 172.21.15.49:0/3968926755 learned_addr learned my addr 172.21.15.49:0/3968926755 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230e64f640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230de4e640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 0x7f230810b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 -- 172.21.15.49:0/3968926755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 msgr2=0x7f2308109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 -- 172.21.15.49:0/3968926755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 msgr2=0x7f230810b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 0x7f230810b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 -- 172.21.15.49:0/3968926755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230810a470 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230e64f640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f230d64d640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009230 tx=0x7f22fc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc041070 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc0044d0 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081a1bd0 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc034440 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.020+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a2120 con 0x7f23081018d0 2024-03-16T08:14:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.021+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2308107a80 con 0x7f23081018d0 2024-03-16T08:14:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.023+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f22fc044070 con 0x7f23081018d0 2024-03-16T08:14:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.023+0000 7f2306ffd640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 0x7f22f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.023+0000 7f230de4e640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 0x7f22f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.024+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22fc0f05f0 con 0x7f23081018d0 2024-03-16T08:14:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.024+0000 7f230de4e640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 0x7f22f00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2308101730 tx=0x7f2300009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:24.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.027+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22fc0bdb30 con 0x7f23081018d0 2024-03-16T08:14:24.069 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:14:24.069 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: Deploying daemon mds.l on smithi154 2024-03-16T08:14:24.070 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:23 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1144285686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:24.175 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: Deploying daemon mds.l on smithi154 2024-03-16T08:14:24.176 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:23 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1144285686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:24.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.212+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f230802d420 con 0x7f23081018d0 2024-03-16T08:14:24.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.212+0000 7f2306ffd640 1 -- 172.21.15.49:0/3968926755 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f22fc0bd430 con 0x7f23081018d0 2024-03-16T08:14:24.214 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 msgr2=0x7f22f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 0x7f22f00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2308101730 tx=0x7f2300009500 comp rx=0 tx=0).stop 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 msgr2=0x7f23081096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009230 tx=0x7f22fc005d50 comp rx=0 tx=0).stop 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 shutdown_connections 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2308102290 0x7f2308109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f22f00a2800 0x7f22f00a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23081018d0 0x7f23081096a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 --2- 172.21.15.49:0/3968926755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23081006e0 0x7f230810b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.214+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 >> 172.21.15.49:0/3968926755 conn(0x7f23080fbc70 msgr2=0x7f2308104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:24.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.215+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 shutdown_connections 2024-03-16T08:14:24.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.215+0000 7f23100b2640 1 -- 172.21.15.49:0/3968926755 wait complete. 2024-03-16T08:14:24.230 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2956244771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98100780 msgr2=0x7fef98100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2956244771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98100780 0x7fef98100ba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef8802bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2956244771 shutdown_connections 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2956244771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98102330 0x7fef98108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2956244771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 0x7fef98101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2956244771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98100780 0x7fef98100ba0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2956244771 >> 172.21.15.49:0/2956244771 conn(0x7fef980fbd10 msgr2=0x7fef980fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2956244771 shutdown_connections 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2956244771 wait complete. 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 Processor -- start 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.339+0000 7fef9e5f3640 1 -- start start 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 0x7fef98197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 0x7fef981980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810c360 con 0x7fef98102330 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810c1e0 con 0x7fef98101970 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810c4e0 con 0x7fef98100780 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38194/0 (socket says 172.21.15.49:38194) 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 -- 172.21.15.49:0/2887844509 learned_addr learned my addr 172.21.15.49:0/2887844509 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef97fff640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 0x7fef98197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef977fe640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 0x7fef981980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 -- 172.21.15.49:0/2887844509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 msgr2=0x7fef98197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 0x7fef98197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 -- 172.21.15.49:0/2887844509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 msgr2=0x7fef981980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 0x7fef981980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 -- 172.21.15.49:0/2887844509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9819f770 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef97fff640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 0x7fef98197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9cb90640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef9000cad0 tx=0x7fef9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000ae20 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef90005020 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9819fa60 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000f6f0 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.340+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9819ff60 con 0x7fef98102330 2024-03-16T08:14:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.341+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef981a01f0 con 0x7fef98102330 2024-03-16T08:14:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.342+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fef900051d0 con 0x7fef98102330 2024-03-16T08:14:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.343+0000 7fef957fa640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 0x7fef800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.343+0000 7fef97fff640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 0x7fef800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.343+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fef900ce710 con 0x7fef98102330 2024-03-16T08:14:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.343+0000 7fef97fff640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 0x7fef800a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fef88036040 tx=0x7fef88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.347+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef90014030 con 0x7fef98102330 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: Deploying daemon mds.l on smithi154 2024-03-16T08:14:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:23 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1144285686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.584+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef981a03c0 con 0x7fef98102330 2024-03-16T08:14:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.584+0000 7fef957fa640 1 -- 172.21.15.49:0/2887844509 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fef9009bc50 con 0x7fef98102330 2024-03-16T08:14:24.585 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:24.586 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-16T08:14:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 1 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.586+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 msgr2=0x7fef800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.586+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 0x7fef800a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fef88036040 tx=0x7fef88000bc0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.586+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 msgr2=0x7fef9819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.586+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef9000cad0 tx=0x7fef9000cf90 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 shutdown_connections 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef800a2800 0x7fef800a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef98102330 0x7fef9819f100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef98101970 0x7fef981980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 --2- 172.21.15.49:0/2887844509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef98100780 0x7fef98197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 >> 172.21.15.49:0/2887844509 conn(0x7fef980fbd10 msgr2=0x7fef98069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 shutdown_connections 2024-03-16T08:14:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:24.587+0000 7fef9e5f3640 1 -- 172.21.15.49:0/2887844509 wait complete. 2024-03-16T08:14:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:25 smithi129 ceph-mon[36561]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3968926755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2887844509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:25.452 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:25 smithi154 ceph-mon[36471]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:25.452 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3968926755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:25.452 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2887844509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:25.453 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 systemd[1]: Starting Ceph mds.l for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:25 smithi049 ceph-mon[31942]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3968926755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2887844509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:25.713 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 podman[67041]: 2024-03-16 08:14:25.355404858 +0000 UTC m=+0.017837021 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:25.713 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 podman[67041]: 2024-03-16 08:14:25.476983503 +0000 UTC m=+0.139415668 container create 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD) 2024-03-16T08:14:25.713 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 podman[67041]: 2024-03-16 08:14:25.713250804 +0000 UTC m=+0.375682968 container init 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-16T08:14:25.975 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 podman[67041]: 2024-03-16 08:14:25.716104387 +0000 UTC m=+0.378536533 container start 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T08:14:25.975 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 ceph-mds[67060]: -- 172.21.15.154:0/2526154971 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba95577a40 con 0x55ba9554ac00 2024-03-16T08:14:25.975 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l[67056]: starting mds.l at 2024-03-16T08:14:25.976 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 bash[67041]: 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de 2024-03-16T08:14:25.976 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:14:25 smithi154 systemd[1]: Started Ceph mds.l for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:26.599 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- 172.21.15.49:0/2354057210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 msgr2=0x7f75e0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 --2- 172.21.15.49:0/2354057210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0101d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009900 tx=0x7f75d002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- 172.21.15.49:0/2354057210 shutdown_connections 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 --2- 172.21.15.49:0/2354057210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e01022b0 0x7f75e0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 --2- 172.21.15.49:0/2354057210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0101d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 --2- 172.21.15.49:0/2354057210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0100700 0x7f75e0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- 172.21.15.49:0/2354057210 >> 172.21.15.49:0/2354057210 conn(0x7f75e00fbc70 msgr2=0x7f75e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- 172.21.15.49:0/2354057210 shutdown_connections 2024-03-16T08:14:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- 172.21.15.49:0/2354057210 wait complete. 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 Processor -- start 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.721+0000 7f75e780c640 1 -- start start 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 0x7f75e0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e01022b0 0x7f75e019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010d4a0 con 0x7f75e01018f0 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010d320 con 0x7f75e0100700 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e780c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010d620 con 0x7f75e01022b0 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38206/0 (socket says 172.21.15.49:38206) 2024-03-16T08:14:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e55a8640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 0x7f75e0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 -- 172.21.15.49:0/2825943420 learned_addr learned my addr 172.21.15.49:0/2825943420 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e5da9640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e01022b0 0x7f75e019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 -- 172.21.15.49:0/2825943420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e01022b0 msgr2=0x7f75e019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e01022b0 0x7f75e019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 -- 172.21.15.49:0/2825943420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 msgr2=0x7f75e0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 0x7f75e0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 -- 172.21.15.49:0/2825943420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e019f720 con 0x7f75e01018f0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e55a8640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 0x7f75e0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75e4da7640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009230 tx=0x7f75d002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.722+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0041070 con 0x7f75e01018f0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.723+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d0004450 con 0x7f75e01018f0 2024-03-16T08:14:26.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.723+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e019f9b0 con 0x7f75e01018f0 2024-03-16T08:14:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.723+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0003e20 con 0x7f75e01018f0 2024-03-16T08:14:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.723+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e019fea0 con 0x7f75e01018f0 2024-03-16T08:14:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.723+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e0107af0 con 0x7f75e01018f0 2024-03-16T08:14:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.725+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f75d0044070 con 0x7f75e01018f0 2024-03-16T08:14:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.725+0000 7f75d67fc640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 0x7f75cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.725+0000 7f75e55a8640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 0x7f75cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.725+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f75d00f0160 con 0x7f75e01018f0 2024-03-16T08:14:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.726+0000 7f75e55a8640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 0x7f75cc0a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75e0101750 tx=0x7f75dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.730+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f75d00bd050 con 0x7f75e01018f0 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6832/866170540,v1:172.21.15.154:6833/866170540] up:boot 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: fsmap 1 up:standby 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.805 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:26.806 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:26.806 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:26.806 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:26 smithi129 ceph-mon[36561]: Deploying daemon mds.k on smithi129 2024-03-16T08:14:26.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.954+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75e002d400 con 0x7f75e01018f0 2024-03-16T08:14:26.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.955+0000 7f75d67fc640 1 -- 172.21.15.49:0/2825943420 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f75d00c10f0 con 0x7f75e01018f0 2024-03-16T08:14:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:26.955 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T08:14:26.955 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 msgr2=0x7f75cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 0x7f75cc0a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75e0101750 tx=0x7f75dc005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 msgr2=0x7f75e0198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009230 tx=0x7f75d002c650 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 shutdown_connections 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e01022b0 0x7f75e019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f75cc0a2800 0x7f75cc0a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e01018f0 0x7f75e0198040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 --2- 172.21.15.49:0/2825943420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75e0100700 0x7f75e0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 >> 172.21.15.49:0/2825943420 conn(0x7f75e00fbc70 msgr2=0x7f75e0104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 shutdown_connections 2024-03-16T08:14:26.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:26.957+0000 7f75e780c640 1 -- 172.21.15.49:0/2825943420 wait complete. 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6832/866170540,v1:172.21.15.154:6833/866170540] up:boot 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: fsmap 1 up:standby 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:26.967 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:26 smithi049 ceph-mon[31942]: Deploying daemon mds.k on smithi129 2024-03-16T08:14:27.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:27.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6832/866170540,v1:172.21.15.154:6833/866170540] up:boot 2024-03-16T08:14:27.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: fsmap 1 up:standby 2024-03-16T08:14:27.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:27.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:26 smithi154 ceph-mon[36471]: Deploying daemon mds.k on smithi129 2024-03-16T08:14:27.921 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:27 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2825943420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:28.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:27 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2825943420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:28.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:27 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2825943420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:28.314 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 systemd[1]: Starting Ceph mds.k for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:28.314 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 podman[66074]: 2024-03-16 08:14:28.216909001 +0000 UTC m=+0.017816753 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:28.610 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 podman[66074]: 2024-03-16 08:14:28.346562963 +0000 UTC m=+0.147470745 container create 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T08:14:28.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:29.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:28 smithi049 ceph-mon[31942]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:29.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 -- 172.21.15.49:0/2820080507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c104c70 msgr2=0x7f643c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 --2- 172.21.15.49:0/2820080507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c104c70 0x7f643c107060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6430009900 tx=0x7f643002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 -- 172.21.15.49:0/2820080507 shutdown_connections 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 --2- 172.21.15.49:0/2820080507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c1075a0 0x7f643c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 --2- 172.21.15.49:0/2820080507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c104c70 0x7f643c107060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 --2- 172.21.15.49:0/2820080507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c102340 0x7f643c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 -- 172.21.15.49:0/2820080507 >> 172.21.15.49:0/2820080507 conn(0x7f643c0fbc70 msgr2=0x7f643c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 -- 172.21.15.49:0/2820080507 shutdown_connections 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.083+0000 7f6440eb2640 1 -- 172.21.15.49:0/2820080507 wait complete. 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 Processor -- start 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 -- start start 2024-03-16T08:14:29.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c104c70 0x7f643c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c1075a0 0x7f643c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d480 con 0x7f643c102340 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d300 con 0x7f643c104c70 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6440eb2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d600 con 0x7f643c1075a0 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f643a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f643a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38218/0 (socket says 172.21.15.49:38218) 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f643a59c640 1 -- 172.21.15.49:0/3193617066 learned_addr learned my addr 172.21.15.49:0/3193617066 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f6439d9b640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c104c70 0x7f643c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.084+0000 7f643ad9d640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c1075a0 0x7f643c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 -- 172.21.15.49:0/3193617066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c1075a0 msgr2=0x7f643c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c1075a0 0x7f643c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 -- 172.21.15.49:0/3193617066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c104c70 msgr2=0x7f643c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c104c70 0x7f643c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 -- 172.21.15.49:0/3193617066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19f7a0 con 0x7f643c102340 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f643a59c640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c970 tx=0x7f642c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c007bb0 con 0x7f643c102340 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642c007d50 con 0x7f643c102340 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19fa90 con 0x7f643c102340 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c00f450 con 0x7f643c102340 2024-03-16T08:14:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.085+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c19ff60 con 0x7f643c102340 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.086+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c0fd760 con 0x7f643c102340 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.088+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f642c004fe0 con 0x7f643c102340 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.089+0000 7f642b7fe640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 0x7f64240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.089+0000 7f6439d9b640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 0x7f64240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.089+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f642c0ce700 con 0x7f643c102340 2024-03-16T08:14:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.089+0000 7f6439d9b640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 0x7f64240a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:29.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.092+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f642c014030 con 0x7f643c102340 2024-03-16T08:14:29.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:28 smithi129 ceph-mon[36561]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:29.110 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 podman[66074]: 2024-03-16 08:14:28.723206624 +0000 UTC m=+0.524114363 container init 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T08:14:29.110 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 podman[66074]: 2024-03-16 08:14:28.725907484 +0000 UTC m=+0.526815226 container start 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T08:14:29.110 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 ceph-mds[66093]: -- 172.21.15.129:0/3170565127 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e82de6ba40 con 0x55e82de3e800 2024-03-16T08:14:29.111 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k[66089]: starting mds.k at 2024-03-16T08:14:29.111 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 bash[66074]: 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 2024-03-16T08:14:29.111 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:14:28 smithi129 systemd[1]: Started Ceph mds.k for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:29.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:28 smithi154 ceph-mon[36471]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:29.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.316+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c02d1b0 con 0x7f643c102340 2024-03-16T08:14:29.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.316+0000 7f642b7fe640 1 -- 172.21.15.49:0/3193617066 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f642c09b5f0 con 0x7f643c102340 2024-03-16T08:14:29.316 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:29.317 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T08:14:29.317 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 msgr2=0x7f64240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 0x7f64240a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430005fd0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 msgr2=0x7f643c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c970 tx=0x7f642c00ce30 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 shutdown_connections 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f643c1075a0 0x7f643c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f643c104c70 0x7f643c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64240a2800 0x7f64240a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 --2- 172.21.15.49:0/3193617066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c102340 0x7f643c197b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 >> 172.21.15.49:0/3193617066 conn(0x7f643c0fbc70 msgr2=0x7f643c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 shutdown_connections 2024-03-16T08:14:29.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:29.318+0000 7f6440eb2640 1 -- 172.21.15.49:0/3193617066 wait complete. 2024-03-16T08:14:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: Deploying daemon mds.j on smithi049 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3193617066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6832/3770218709,v1:172.21.15.129:6833/3770218709] up:boot 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: fsmap 2 up:standby 2024-03-16T08:14:30.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:29 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:14:30.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: Deploying daemon mds.j on smithi049 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3193617066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6832/3770218709,v1:172.21.15.129:6833/3770218709] up:boot 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: fsmap 2 up:standby 2024-03-16T08:14:30.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:29 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:14:30.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:30.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:30.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: Deploying daemon mds.j on smithi049 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3193617066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6832/3770218709,v1:172.21.15.129:6833/3770218709] up:boot 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: fsmap 2 up:standby 2024-03-16T08:14:30.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:29 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:14:31.129 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:30 smithi129 ceph-mon[36561]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:31.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:30 smithi154 ceph-mon[36471]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:31.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:30 smithi049 ceph-mon[31942]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:31.288 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 systemd[1]: Starting Ceph mds.j for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:31.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:31.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 -- 172.21.15.49:0/1902991173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b01018f0 msgr2=0x7f80b0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:31.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1902991173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b01018f0 0x7f80b0101d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009900 tx=0x7f80a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:31.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 -- 172.21.15.49:0/1902991173 shutdown_connections 2024-03-16T08:14:31.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1902991173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 0x7f80b0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1902991173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b01018f0 0x7f80b0101d70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1902991173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0100700 0x7f80b0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 -- 172.21.15.49:0/1902991173 >> 172.21.15.49:0/1902991173 conn(0x7f80b00fbc70 msgr2=0x7f80b00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:31.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 -- 172.21.15.49:0/1902991173 shutdown_connections 2024-03-16T08:14:31.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.445+0000 7f80b58fe640 1 -- 172.21.15.49:0/1902991173 wait complete. 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 Processor -- start 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 -- start start 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 0x7f80b0102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 0x7f80b0103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010c380 con 0x7f80b0100700 2024-03-16T08:14:31.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010c200 con 0x7f80b01018f0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80b58fe640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010c500 con 0x7f80b01022b0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38232/0 (socket says 172.21.15.49:38232) 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80aeffd640 1 -- 172.21.15.49:0/1831623156 learned_addr learned my addr 172.21.15.49:0/1831623156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.446+0000 7f80ae7fc640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 0x7f80b0102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80af7fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 0x7f80b0103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 -- 172.21.15.49:0/1831623156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 msgr2=0x7f80b0103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 0x7f80b0103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 -- 172.21.15.49:0/1831623156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 msgr2=0x7f80b0102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 0x7f80b0102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 -- 172.21.15.49:0/1831623156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0103aa0 con 0x7f80b0100700 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80ae7fc640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 0x7f80b0102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80aeffd640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ebc0 tx=0x7f80a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0003b70 con 0x7f80b0100700 2024-03-16T08:14:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b01a1c40 con 0x7f80b0100700 2024-03-16T08:14:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a0003d10 con 0x7f80b0100700 2024-03-16T08:14:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0019980 con 0x7f80b0100700 2024-03-16T08:14:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.447+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a21b0 con 0x7f80b0100700 2024-03-16T08:14:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.448+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b01a2380 con 0x7f80b0100700 2024-03-16T08:14:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.450+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f80a0019b20 con 0x7f80b0100700 2024-03-16T08:14:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.450+0000 7f80b48fc640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 0x7f80980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.451+0000 7f80ae7fc640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 0x7f80980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.451+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80a00ceed0 con 0x7f80b0100700 2024-03-16T08:14:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.451+0000 7f80ae7fc640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 0x7f80980a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f80a402c7b0 tx=0x7f80a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.454+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f80a009c4a0 con 0x7f80b0100700 2024-03-16T08:14:31.579 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 podman[71966]: 2024-03-16 08:14:31.228292633 +0000 UTC m=+0.018165858 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:31.579 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 podman[71966]: 2024-03-16 08:14:31.35159402 +0000 UTC m=+0.141467232 container create a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T08:14:31.580 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 podman[71966]: 2024-03-16 08:14:31.579864053 +0000 UTC m=+0.369737269 container init a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-16T08:14:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.680+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80b01a2550 con 0x7f80b0100700 2024-03-16T08:14:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.681+0000 7f80b48fc640 1 -- 172.21.15.49:0/1831623156 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f80a009bda0 con 0x7f80b0100700 2024-03-16T08:14:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:31.681 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-16T08:14:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-03-16T08:14:31.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 msgr2=0x7f80980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 0x7f80980a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f80a402c7b0 tx=0x7f80a4005fd0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 msgr2=0x7f80b0104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ebc0 tx=0x7f80a000c6a0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 shutdown_connections 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01022b0 0x7f80b0103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01018f0 0x7f80b0102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f80980a2800 0x7f80980a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 --2- 172.21.15.49:0/1831623156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b0100700 0x7f80b0104690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 >> 172.21.15.49:0/1831623156 conn(0x7f80b00fbc70 msgr2=0x7f80b00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.683+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 shutdown_connections 2024-03-16T08:14:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:31.684+0000 7f80b58fe640 1 -- 172.21.15.49:0/1831623156 wait complete. 2024-03-16T08:14:31.915 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 podman[71966]: 2024-03-16 08:14:31.583515382 +0000 UTC m=+0.373388790 container start a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:14:31.915 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mds[72024]: -- 172.21.15.49:0/610684556 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56327e099a40 con 0x56327e06cc00 2024-03-16T08:14:31.915 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j[72020]: starting mds.j at 2024-03-16T08:14:31.915 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 bash[71966]: a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 2024-03-16T08:14:31.915 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:14:31 smithi049 systemd[1]: Started Ceph mds.j for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:32.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1831623156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:32.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:31 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1831623156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:32.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:31 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1831623156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:32.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:32.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:32.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:31 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:33.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:32 smithi049 ceph-mon[31942]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:33.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:32 smithi049 ceph-mon[31942]: Deploying daemon mds.d on smithi049 2024-03-16T08:14:33.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:32 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.49:6834/2399676198,v1:172.21.15.49:6835/2399676198] up:boot 2024-03-16T08:14:33.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:32 smithi049 ceph-mon[31942]: fsmap 3 up:standby 2024-03-16T08:14:33.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:32 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:14:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:32 smithi154 ceph-mon[36471]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:32 smithi154 ceph-mon[36471]: Deploying daemon mds.d on smithi049 2024-03-16T08:14:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:32 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.49:6834/2399676198,v1:172.21.15.49:6835/2399676198] up:boot 2024-03-16T08:14:33.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:32 smithi154 ceph-mon[36471]: fsmap 3 up:standby 2024-03-16T08:14:33.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:32 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:14:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:32 smithi129 ceph-mon[36561]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:33.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:32 smithi129 ceph-mon[36561]: Deploying daemon mds.d on smithi049 2024-03-16T08:14:33.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:32 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.49:6834/2399676198,v1:172.21.15.49:6835/2399676198] up:boot 2024-03-16T08:14:33.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:32 smithi129 ceph-mon[36561]: fsmap 3 up:standby 2024-03-16T08:14:33.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:32 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:14:33.697 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 -- 172.21.15.49:0/2114461229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 msgr2=0x7f5a7c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 --2- 172.21.15.49:0/2114461229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c109c50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009ca0 tx=0x7f5a7402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 -- 172.21.15.49:0/2114461229 shutdown_connections 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 --2- 172.21.15.49:0/2114461229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c109c50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 --2- 172.21.15.49:0/2114461229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c102a00 0x7f5a7c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 --2- 172.21.15.49:0/2114461229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c101810 0x7f5a7c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 -- 172.21.15.49:0/2114461229 >> 172.21.15.49:0/2114461229 conn(0x7f5a7c0fcd80 msgr2=0x7f5a7c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:33.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 -- 172.21.15.49:0/2114461229 shutdown_connections 2024-03-16T08:14:33.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.825+0000 7f5a83edc640 1 -- 172.21.15.49:0/2114461229 wait complete. 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 Processor -- start 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 -- start start 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 0x7f5a7c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 0x7f5a7c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d470 con 0x7f5a7c1033c0 2024-03-16T08:14:33.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d2f0 con 0x7f5a7c102a00 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a83edc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d5f0 con 0x7f5a7c101810 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57004/0 (socket says 172.21.15.49:57004) 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 -- 172.21.15.49:0/1875550167 learned_addr learned my addr 172.21.15.49:0/1875550167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a81c78640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 0x7f5a7c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a81477640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 0x7f5a7c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 -- 172.21.15.49:0/1875550167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 msgr2=0x7f5a7c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 0x7f5a7c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 -- 172.21.15.49:0/1875550167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 msgr2=0x7f5a7c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 0x7f5a7c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a82479640 1 -- 172.21.15.49:0/1875550167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c19f640 con 0x7f5a7c1033c0 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a81c78640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 0x7f5a7c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:14:33.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.826+0000 7f5a81477640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 0x7f5a7c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a82479640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5a7402c700 tx=0x7f5a74032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74039040 con 0x7f5a7c1033c0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a74044070 con 0x7f5a7c1033c0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c19f8d0 con 0x7f5a7c1033c0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740042a0 con 0x7f5a7c1033c0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.827+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c19fe10 con 0x7f5a7c1033c0 2024-03-16T08:14:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.828+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c108ad0 con 0x7f5a7c1033c0 2024-03-16T08:14:33.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.829+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f5a74003970 con 0x7f5a7c1033c0 2024-03-16T08:14:33.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.830+0000 7f5a72ffd640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 0x7f5a680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:33.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.830+0000 7f5a81c78640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 0x7f5a680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:33.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.830+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5a740f1100 con 0x7f5a7c1033c0 2024-03-16T08:14:33.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.830+0000 7f5a81c78640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 0x7f5a680a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c102860 tx=0x7f5a6c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:33.838+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a740be6d0 con 0x7f5a7c1033c0 2024-03-16T08:14:34.037 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:33 smithi049 systemd[1]: Starting Ceph mds.d for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:34.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.074+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a7c02d1b0 con 0x7f5a7c1033c0 2024-03-16T08:14:34.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.074+0000 7f5a72ffd640 1 -- 172.21.15.49:0/1875550167 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f5a74047e40 con 0x7f5a7c1033c0 2024-03-16T08:14:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:34.075 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-16T08:14:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 4 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 msgr2=0x7f5a680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 0x7f5a680a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c102860 tx=0x7f5a6c005e70 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 msgr2=0x7f5a7c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5a7402c700 tx=0x7f5a74032f20 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 shutdown_connections 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a680a2880 0x7f5a680a4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7c1033c0 0x7f5a7c19efd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7c102a00 0x7f5a7c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 --2- 172.21.15.49:0/1875550167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a7c101810 0x7f5a7c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.076+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 >> 172.21.15.49:0/1875550167 conn(0x7f5a7c0fcd80 msgr2=0x7f5a7c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.077+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 shutdown_connections 2024-03-16T08:14:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:34.077+0000 7f5a83edc640 1 -- 172.21.15.49:0/1875550167 wait complete. 2024-03-16T08:14:34.373 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 podman[72356]: 2024-03-16 08:14:34.088136772 +0000 UTC m=+0.019445995 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:34.373 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 podman[72356]: 2024-03-16 08:14:34.191883221 +0000 UTC m=+0.123192430 container create 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:14:34.787 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 podman[72356]: 2024-03-16 08:14:34.428269199 +0000 UTC m=+0.359578403 container init 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-16T08:14:34.788 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 podman[72356]: 2024-03-16 08:14:34.431866986 +0000 UTC m=+0.363176191 container start 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:14:34.788 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mds[72378]: -- 172.21.15.49:0/3690042637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560122267a40 con 0x56012223a800 2024-03-16T08:14:34.788 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d[72374]: starting mds.d at 2024-03-16T08:14:34.788 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 bash[72356]: 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b 2024-03-16T08:14:34.788 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:14:34 smithi049 systemd[1]: Started Ceph mds.d for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1875550167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:35.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:35.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:34 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1875550167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:35.239 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:34 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1875550167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:34 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:36.091 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:36.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 -- 172.21.15.49:0/1923639563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41029f0 msgr2=0x7f0ff4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/1923639563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41029f0 0x7f0ff4102e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009900 tx=0x7f0fe802bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 -- 172.21.15.49:0/1923639563 shutdown_connections 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/1923639563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41033b0 0x7f0ff4109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/1923639563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41029f0 0x7f0ff4102e70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/1923639563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 0x7f0ff4101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 -- 172.21.15.49:0/1923639563 >> 172.21.15.49:0/1923639563 conn(0x7f0ff40fcd90 msgr2=0x7f0ff40ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 -- 172.21.15.49:0/1923639563 shutdown_connections 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.216+0000 7f0ffbc59640 1 -- 172.21.15.49:0/1923639563 wait complete. 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 Processor -- start 2024-03-16T08:14:36.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 -- start start 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 0x7f0ff4075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41029f0 0x7f0ff4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410d460 con 0x7f0ff41033b0 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410d2e0 con 0x7f0ff41029f0 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffbc59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410d5e0 con 0x7f0ff4101800 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57038/0 (socket says 172.21.15.49:57038) 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 -- 172.21.15.49:0/3266133551 learned_addr learned my addr 172.21.15.49:0/3266133551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ff99f5640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 0x7f0ff4075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ff91f4640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41029f0 0x7f0ff4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:36.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 -- 172.21.15.49:0/3266133551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 msgr2=0x7f0ff4075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 0x7f0ff4075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 -- 172.21.15.49:0/3266133551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41029f0 msgr2=0x7f0ff4073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41029f0 0x7f0ff4073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 -- 172.21.15.49:0/3266133551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff40749a0 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.217+0000 7f0ffa1f6640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00d750 tx=0x7f0fec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.218+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec017070 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.218+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fec005500 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.218+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec004d90 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.218+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff41a1b90 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.218+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff41a2060 con 0x7f0ff41033b0 2024-03-16T08:14:36.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.219+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff4108ac0 con 0x7f0ff41033b0 2024-03-16T08:14:36.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.220+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0fec004f30 con 0x7f0ff41033b0 2024-03-16T08:14:36.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.220+0000 7f0ff2ffd640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:36.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.220+0000 7f0ff99f5640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:36.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.220+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0fec0ce000 con 0x7f0ff41033b0 2024-03-16T08:14:36.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.221+0000 7f0ff99f5640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005c40 tx=0x7f0fe4005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.225+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0fec09b540 con 0x7f0ff41033b0 2024-03-16T08:14:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:35 smithi154 ceph-mon[36471]: Deploying daemon mds.e on smithi129 2024-03-16T08:14:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:35 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.49:6836/761407043,v1:172.21.15.49:6837/761407043] up:boot 2024-03-16T08:14:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:35 smithi154 ceph-mon[36471]: fsmap 4 up:standby 2024-03-16T08:14:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:35 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:14:36.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:35 smithi129 ceph-mon[36561]: Deploying daemon mds.e on smithi129 2024-03-16T08:14:36.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:35 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.49:6836/761407043,v1:172.21.15.49:6837/761407043] up:boot 2024-03-16T08:14:36.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:35 smithi129 ceph-mon[36561]: fsmap 4 up:standby 2024-03-16T08:14:36.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:35 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:14:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:35 smithi049 ceph-mon[31942]: Deploying daemon mds.e on smithi129 2024-03-16T08:14:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:35 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.49:6836/761407043,v1:172.21.15.49:6837/761407043] up:boot 2024-03-16T08:14:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:35 smithi049 ceph-mon[31942]: fsmap 4 up:standby 2024-03-16T08:14:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:35 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:14:36.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.449+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ff402d1b0 con 0x7f0ff41033b0 2024-03-16T08:14:36.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.449+0000 7f0ff2ffd640 1 -- 172.21.15.49:0/3266133551 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f0fec09ae40 con 0x7f0ff41033b0 2024-03-16T08:14:36.449 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:36.450 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-16T08:14:36.450 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2024-03-16T08:14:36.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 msgr2=0x7f0fdc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:36.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005c40 tx=0x7f0fe4005bd0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 msgr2=0x7f0ff4074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00d750 tx=0x7f0fec00dc10 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 shutdown_connections 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ff41033b0 0x7f0ff4074110 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ff41029f0 0x7f0ff4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 --2- 172.21.15.49:0/3266133551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff4101800 0x7f0ff4075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 >> 172.21.15.49:0/3266133551 conn(0x7f0ff40fcd90 msgr2=0x7f0ff4105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 shutdown_connections 2024-03-16T08:14:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:36.451+0000 7f0ffbc59640 1 -- 172.21.15.49:0/3266133551 wait complete. 2024-03-16T08:14:36.861 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:36 smithi129 systemd[1]: Starting Ceph mds.e for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:37.210 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:36 smithi129 ceph-mon[36561]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:37.211 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:36 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3266133551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:37.211 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:36 smithi129 podman[66395]: 2024-03-16 08:14:36.835155297 +0000 UTC m=+0.018156036 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:37.211 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:36 smithi129 podman[66395]: 2024-03-16 08:14:36.966449684 +0000 UTC m=+0.149450422 container create de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-16T08:14:37.211 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 podman[66395]: 2024-03-16 08:14:37.210926962 +0000 UTC m=+0.393927709 container init de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:14:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:36 smithi154 ceph-mon[36471]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:37.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:36 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3266133551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:36 smithi049 ceph-mon[31942]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:37.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:36 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3266133551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:37.610 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 podman[66395]: 2024-03-16 08:14:37.213877413 +0000 UTC m=+0.396878147 container start de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-16T08:14:37.611 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 ceph-mds[66414]: -- 172.21.15.129:0/3383037224 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610cb455a40 con 0x5610cb428c00 2024-03-16T08:14:37.611 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: starting mds.e at 2024-03-16T08:14:37.611 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 bash[66395]: de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 2024-03-16T08:14:37.611 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:14:37 smithi129 systemd[1]: Started Ceph mds.e for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:38.463 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 -- 172.21.15.49:0/2783524182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c104c90 msgr2=0x7f493c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 --2- 172.21.15.49:0/2783524182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c104c90 0x7f493c107080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4924009900 tx=0x7f492402bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 -- 172.21.15.49:0/2783524182 shutdown_connections 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 --2- 172.21.15.49:0/2783524182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c1075c0 0x7f493c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 --2- 172.21.15.49:0/2783524182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c104c90 0x7f493c107080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 --2- 172.21.15.49:0/2783524182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 0x7f493c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.583+0000 7f4941485640 1 -- 172.21.15.49:0/2783524182 >> 172.21.15.49:0/2783524182 conn(0x7f493c0fbd10 msgr2=0x7f493c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- 172.21.15.49:0/2783524182 shutdown_connections 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- 172.21.15.49:0/2783524182 wait complete. 2024-03-16T08:14:38.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 Processor -- start 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- start start 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 0x7f493c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c104c90 0x7f493c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f493c10d480 con 0x7f493c1075c0 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f493c10d300 con 0x7f493c104c90 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f4941485640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f493c10d600 con 0x7f493c102360 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f493b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f493b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57064/0 (socket says 172.21.15.49:57064) 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f493b7fe640 1 -- 172.21.15.49:0/618671046 learned_addr learned my addr 172.21.15.49:0/618671046 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.584+0000 7f493affd640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 0x7f493c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4933fff640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c104c90 0x7f493c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 -- 172.21.15.49:0/618671046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 msgr2=0x7f493c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 0x7f493c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 -- 172.21.15.49:0/618671046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c104c90 msgr2=0x7f493c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c104c90 0x7f493c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 -- 172.21.15.49:0/618671046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c19f750 con 0x7f493c1075c0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f493b7fe640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f493400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934017070 con 0x7f493c1075c0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934003e50 con 0x7f493c1075c0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c19fa40 con 0x7f493c1075c0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340106c0 con 0x7f493c1075c0 2024-03-16T08:14:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.585+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c19ff60 con 0x7f493c1075c0 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.586+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c0fd800 con 0x7f493c1075c0 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.587+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4934003970 con 0x7f493c1075c0 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.588+0000 7f4938ff9640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 0x7f49280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.588+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f49340ce260 con 0x7f493c1075c0 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.588+0000 7f493affd640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 0x7f49280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.588+0000 7f493affd640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 0x7f49280a4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f492c009bd0 tx=0x7f492c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:38.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.592+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f493409b150 con 0x7f493c1075c0 2024-03-16T08:14:38.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: Deploying daemon mds.f on smithi154 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6834/272945510,v1:172.21.15.129:6835/272945510] up:boot 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: fsmap 5 up:standby 2024-03-16T08:14:38.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:38 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:14:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: Deploying daemon mds.f on smithi154 2024-03-16T08:14:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6834/272945510,v1:172.21.15.129:6835/272945510] up:boot 2024-03-16T08:14:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: fsmap 5 up:standby 2024-03-16T08:14:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:38 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:14:38.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.814+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f493c0623e0 con 0x7f493c1075c0 2024-03-16T08:14:38.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.815+0000 7f4938ff9640 1 -- 172.21.15.49:0/618671046 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f493409f1f0 con 0x7f493c1075c0 2024-03-16T08:14:38.815 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:38.815 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-16T08:14:38.815 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 6 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 msgr2=0x7f49280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 0x7f49280a4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f492c009bd0 tx=0x7f492c005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 msgr2=0x7f493c19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f493400b260 comp rx=0 tx=0).stop 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 shutdown_connections 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49280a2880 0x7f49280a4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f493c1075c0 0x7f493c19f0e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f493c104c90 0x7f493c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 --2- 172.21.15.49:0/618671046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102360 0x7f493c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:38.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 >> 172.21.15.49:0/618671046 conn(0x7f493c0fbd10 msgr2=0x7f493c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:38.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 shutdown_connections 2024-03-16T08:14:38.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:38.817+0000 7f4941485640 1 -- 172.21.15.49:0/618671046 wait complete. 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: Deploying daemon mds.f on smithi154 2024-03-16T08:14:38.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6834/272945510,v1:172.21.15.129:6835/272945510] up:boot 2024-03-16T08:14:38.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: fsmap 5 up:standby 2024-03-16T08:14:38.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:38 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:14:39.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:39 smithi154 ceph-mon[36471]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:39.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:39 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/618671046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:39.726 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:39 smithi154 systemd[1]: Starting Ceph mds.f for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:39.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:39 smithi049 ceph-mon[31942]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:39.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:39 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/618671046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:39 smithi129 ceph-mon[36561]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:39 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/618671046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:40.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:39 smithi154 podman[67351]: 2024-03-16 08:14:39.688444381 +0000 UTC m=+0.017826516 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:40.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:39 smithi154 podman[67351]: 2024-03-16 08:14:39.815464959 +0000 UTC m=+0.144847094 container create d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T08:14:40.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 podman[67351]: 2024-03-16 08:14:40.08515514 +0000 UTC m=+0.414537277 container init d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:14:40.475 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 podman[67351]: 2024-03-16 08:14:40.087659016 +0000 UTC m=+0.417041143 container start d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T08:14:40.475 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 ceph-mds[67370]: -- 172.21.15.154:0/1582590689 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606c0a11a40 con 0x5606c09e4c00 2024-03-16T08:14:40.475 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: starting mds.f at 2024-03-16T08:14:40.476 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 bash[67351]: d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 2024-03-16T08:14:40.476 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:14:40 smithi154 systemd[1]: Started Ceph mds.f for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:40.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.949+0000 7f8fef59e640 1 -- 172.21.15.49:0/1221643890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff0100530 msgr2=0x7f8ff0100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.949+0000 7f8fef59e640 1 --2- 172.21.15.49:0/1221643890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff0100530 0x7f8ff0100950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009900 tx=0x7f8fe002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 -- 172.21.15.49:0/1221643890 shutdown_connections 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 --2- 172.21.15.49:0/1221643890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff01020e0 0x7f8ff0108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 --2- 172.21.15.49:0/1221643890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 --2- 172.21.15.49:0/1221643890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff0100530 0x7f8ff0100950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 -- 172.21.15.49:0/1221643890 >> 172.21.15.49:0/1221643890 conn(0x7f8ff00fbac0 msgr2=0x7f8ff00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 -- 172.21.15.49:0/1221643890 shutdown_connections 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 -- 172.21.15.49:0/1221643890 wait complete. 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 Processor -- start 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.950+0000 7f8fef59e640 1 -- start start 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 0x7f8ff00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010c1c0 con 0x7f8ff01020e0 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010c040 con 0x7f8ff0101720 2024-03-16T08:14:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010c340 con 0x7f8ff0100530 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 0x7f8ff00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57084/0 (socket says 172.21.15.49:57084) 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 0x7f8ff00694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37474/0 (socket says 172.21.15.49:37474) 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 -- 172.21.15.49:0/3912299616 learned_addr learned my addr 172.21.15.49:0/3912299616 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fe7fff640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 -- 172.21.15.49:0/3912299616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 msgr2=0x7f8ff00694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 0x7f8ff00694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 -- 172.21.15.49:0/3912299616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 msgr2=0x7f8ff019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 -- 172.21.15.49:0/3912299616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff01a17b0 con 0x7f8ff01020e0 2024-03-16T08:14:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fe7fff640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8feed9d640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800eaa0 tx=0x7f8fe800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800cdb0 con 0x7f8ff01020e0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff01a1a40 con 0x7f8ff01020e0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe8005020 con 0x7f8ff01020e0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80116b0 con 0x7f8ff01020e0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.951+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a1f90 con 0x7f8ff01020e0 2024-03-16T08:14:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.952+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff0107850 con 0x7f8ff01020e0 2024-03-16T08:14:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.954+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8fe8005560 con 0x7f8ff01020e0 2024-03-16T08:14:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.954+0000 7f8fe77fe640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 0x7f8fdc0a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.955+0000 7f8fee59c640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 0x7f8fdc0a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.955+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fe80cebe0 con 0x7f8ff01020e0 2024-03-16T08:14:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.955+0000 7f8fee59c640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 0x7f8fdc0a4cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0000c00 tx=0x7f8fe0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:40.958+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8fe8016030 con 0x7f8ff01020e0 2024-03-16T08:14:41.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.179+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ff002d3b0 con 0x7f8ff01020e0 2024-03-16T08:14:41.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.180+0000 7f8fe77fe640 1 -- 172.21.15.49:0/3912299616 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f8fe809c120 con 0x7f8ff01020e0 2024-03-16T08:14:41.181 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:41.181 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-16T08:14:41.181 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 msgr2=0x7f8fdc0a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 0x7f8fdc0a4cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0000c00 tx=0x7f8fe0005fd0 comp rx=0 tx=0).stop 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 msgr2=0x7f8ff01a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800eaa0 tx=0x7f8fe800ef60 comp rx=0 tx=0).stop 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 shutdown_connections 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f8fdc0a2830 0x7f8fdc0a4cf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:41.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff01020e0 0x7f8ff01a0fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:41.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ff0101720 0x7f8ff019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:41.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 --2- 172.21.15.49:0/3912299616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff0100530 0x7f8ff00694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:41.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 >> 172.21.15.49:0/3912299616 conn(0x7f8ff00fbac0 msgr2=0x7f8ff0104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:41.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 shutdown_connections 2024-03-16T08:14:41.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:41.182+0000 7f8fef59e640 1 -- 172.21.15.49:0/3912299616 wait complete. 2024-03-16T08:14:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: Deploying daemon mds.c on smithi154 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6834/2047612746,v1:172.21.15.154:6835/2047612746] up:boot 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: fsmap 6 up:standby 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:14:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:41 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3912299616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: Deploying daemon mds.c on smithi154 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6834/2047612746,v1:172.21.15.154:6835/2047612746] up:boot 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: fsmap 6 up:standby 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:14:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:41 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3912299616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:41.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:41.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: Deploying daemon mds.c on smithi154 2024-03-16T08:14:41.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6834/2047612746,v1:172.21.15.154:6835/2047612746] up:boot 2024-03-16T08:14:41.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: fsmap 6 up:standby 2024-03-16T08:14:41.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:14:41.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:41 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3912299616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:42.703 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:42 smithi154 systemd[1]: Starting Ceph mds.c for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:42.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:42 smithi154 podman[67655]: 2024-03-16 08:14:42.605529045 +0000 UTC m=+0.017550231 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:42.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:42 smithi154 podman[67655]: 2024-03-16 08:14:42.704655329 +0000 UTC m=+0.116676546 container create 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T08:14:43.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:42 smithi154 podman[67655]: 2024-03-16 08:14:42.982817004 +0000 UTC m=+0.394838194 container init 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:42 smithi154 podman[67655]: 2024-03-16 08:14:42.985363104 +0000 UTC m=+0.397384277 container start 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mds[67674]: -- 172.21.15.154:0/1776721104 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dccddb9a40 con 0x55dccdd8d000 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: starting mds.c at 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:43 smithi154 bash[67655]: 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a 2024-03-16T08:14:43.241 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:14:43 smithi154 systemd[1]: Started Ceph mds.c for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 -- 172.21.15.49:0/3539693702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744101970 msgr2=0x7f7744101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3539693702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744101970 0x7f7744101df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7738009900 tx=0x7f773802bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 -- 172.21.15.49:0/3539693702 shutdown_connections 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3539693702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744102330 0x7f7744108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3539693702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744101970 0x7f7744101df0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3539693702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744100780 0x7f7744100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 -- 172.21.15.49:0/3539693702 >> 172.21.15.49:0/3539693702 conn(0x7f77440fbd10 msgr2=0x7f77440fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 -- 172.21.15.49:0/3539693702 shutdown_connections 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.312+0000 7f774bbb3640 1 -- 172.21.15.49:0/3539693702 wait complete. 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 Processor -- start 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 -- start start 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 0x7f7744197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744101970 0x7f7744198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f774410c390 con 0x7f7744102330 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f774410c210 con 0x7f7744101970 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774bbb3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f774410c510 con 0x7f7744100780 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774a150640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774a150640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41828/0 (socket says 172.21.15.49:41828) 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774994f640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 0x7f7744197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774a150640 1 -- 172.21.15.49:0/3993850115 learned_addr learned my addr 172.21.15.49:0/3993850115 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.313+0000 7f774914e640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744101970 0x7f7744198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 -- 172.21.15.49:0/3993850115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 msgr2=0x7f7744197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 0x7f7744197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 -- 172.21.15.49:0/3993850115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744101970 msgr2=0x7f7744198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744101970 0x7f7744198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 -- 172.21.15.49:0/3993850115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774419f720 con 0x7f7744102330 2024-03-16T08:14:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774994f640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 0x7f7744197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774a150640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f773c00eb10 tx=0x7f773c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c004b20 con 0x7f7744102330 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773c004cc0 con 0x7f7744102330 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774419fa10 con 0x7f7744102330 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c010760 con 0x7f7744102330 2024-03-16T08:14:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.314+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774419ff80 con 0x7f7744102330 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.315+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77441a0210 con 0x7f7744102330 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.317+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f773c005000 con 0x7f7744102330 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.317+0000 7f7736ffd640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 0x7f77300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.318+0000 7f774994f640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 0x7f77300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.318+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f773c0cefc0 con 0x7f7744102330 2024-03-16T08:14:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.318+0000 7f774994f640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 0x7f77300a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7740005fd0 tx=0x7f774000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.321+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f773c09c460 con 0x7f7744102330 2024-03-16T08:14:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:43.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:43.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:43 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:43.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.543+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77441a03e0 con 0x7f7744102330 2024-03-16T08:14:43.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.543+0000 7f7736ffd640 1 -- 172.21.15.49:0/3993850115 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f773c09bd60 con 0x7f7744102330 2024-03-16T08:14:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:43.544 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-16T08:14:43.545 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-03-16T08:14:43.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 msgr2=0x7f77300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:43.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 0x7f77300a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7740005fd0 tx=0x7f774000b410 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 msgr2=0x7f774419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f773c00eb10 tx=0x7f773c00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 shutdown_connections 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f77300a2880 0x7f77300a4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7744102330 0x7f774419f0b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7744101970 0x7f7744198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 --2- 172.21.15.49:0/3993850115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7744100780 0x7f7744197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 >> 172.21.15.49:0/3993850115 conn(0x7f77440fbd10 msgr2=0x7f77440fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 shutdown_connections 2024-03-16T08:14:43.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:43.545+0000 7f774bbb3640 1 -- 172.21.15.49:0/3993850115 wait complete. 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:43.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:43.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:43 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:43.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:43.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:43.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:43.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:43.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:43 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:44 smithi129 ceph-mon[36561]: Deploying daemon mds.h on smithi129 2024-03-16T08:14:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:44 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6836/4234449434,v1:172.21.15.154:6837/4234449434] up:boot 2024-03-16T08:14:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:44 smithi129 ceph-mon[36561]: fsmap 7 up:standby 2024-03-16T08:14:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:44 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:14:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:44 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3993850115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:44.512 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:44 smithi154 ceph-mon[36471]: Deploying daemon mds.h on smithi129 2024-03-16T08:14:44.512 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:44 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6836/4234449434,v1:172.21.15.154:6837/4234449434] up:boot 2024-03-16T08:14:44.512 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:44 smithi154 ceph-mon[36471]: fsmap 7 up:standby 2024-03-16T08:14:44.512 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:44 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:14:44.512 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:44 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3993850115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:44 smithi049 ceph-mon[31942]: Deploying daemon mds.h on smithi129 2024-03-16T08:14:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:44 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6836/4234449434,v1:172.21.15.154:6837/4234449434] up:boot 2024-03-16T08:14:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:44 smithi049 ceph-mon[31942]: fsmap 7 up:standby 2024-03-16T08:14:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:44 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:14:44.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:44 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3993850115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:45.394 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:45 smithi129 ceph-mon[36561]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:45.394 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 systemd[1]: Starting Ceph mds.h for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:45.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:45 smithi049 ceph-mon[31942]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:45.558 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 -- 172.21.15.49:0/768755722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b64100720 msgr2=0x7f0b64100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/768755722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b64100720 0x7f0b64100b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009900 tx=0x7f0b5802bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 -- 172.21.15.49:0/768755722 shutdown_connections 2024-03-16T08:14:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/768755722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b641022d0 0x7f0b64108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/768755722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b64101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/768755722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b64100720 0x7f0b64100b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 -- 172.21.15.49:0/768755722 >> 172.21.15.49:0/768755722 conn(0x7f0b640fbc90 msgr2=0x7f0b640fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 -- 172.21.15.49:0/768755722 shutdown_connections 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.673+0000 7f0b6ae44640 1 -- 172.21.15.49:0/768755722 wait complete. 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 Processor -- start 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 -- start start 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b64100720 0x7f0b6410b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b641096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410d4e0 con 0x7f0b641022d0 2024-03-16T08:14:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410d360 con 0x7f0b64101910 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b6ae44640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410d660 con 0x7f0b64100720 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41850/0 (socket says 172.21.15.49:41850) 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 -- 172.21.15.49:0/3172807156 learned_addr learned my addr 172.21.15.49:0/3172807156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b68be0640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b64100720 0x7f0b6410b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b63fff640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b641096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 -- 172.21.15.49:0/3172807156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b64100720 msgr2=0x7f0b6410b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b64100720 0x7f0b6410b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 -- 172.21.15.49:0/3172807156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 msgr2=0x7f0b641096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b641096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b693e1640 1 -- 172.21.15.49:0/3172807156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6410a4b0 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.674+0000 7f0b63fff640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b693e1640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009fd0 tx=0x7f0b5c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641a1c80 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b5c003e50 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0106c0 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.675+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a21d0 con 0x7f0b641022d0 2024-03-16T08:14:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.676+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b64107a40 con 0x7f0b641022d0 2024-03-16T08:14:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.677+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0b5c003970 con 0x7f0b641022d0 2024-03-16T08:14:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.678+0000 7f0b61ffb640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 0x7f0b540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.678+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0b5c0ce2e0 con 0x7f0b641022d0 2024-03-16T08:14:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.678+0000 7f0b68be0640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 0x7f0b540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.678+0000 7f0b68be0640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 0x7f0b540a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b5802c170 tx=0x7f0b5802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:45.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.682+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0b5c09b820 con 0x7f0b641022d0 2024-03-16T08:14:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:45 smithi154 ceph-mon[36471]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:45.742 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 podman[66702]: 2024-03-16 08:14:45.511901341 +0000 UTC m=+0.017931589 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:45.743 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 podman[66702]: 2024-03-16 08:14:45.630269675 +0000 UTC m=+0.136299915 container create 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:14:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.906+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b6402d3b0 con 0x7f0b641022d0 2024-03-16T08:14:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.906+0000 7f0b61ffb640 1 -- 172.21.15.49:0/3172807156 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f0b5c09b120 con 0x7f0b641022d0 2024-03-16T08:14:45.908 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:45.908 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-16T08:14:45.908 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-03-16T08:14:45.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 msgr2=0x7f0b540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:45.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 0x7f0b540a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b5802c170 tx=0x7f0b5802df90 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 msgr2=0x7f0b64109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009fd0 tx=0x7f0b5c00b260 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 shutdown_connections 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f0b540a2800 0x7f0b540a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b641022d0 0x7f0b64109c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b64101910 0x7f0b641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 --2- 172.21.15.49:0/3172807156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b64100720 0x7f0b6410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.908+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 >> 172.21.15.49:0/3172807156 conn(0x7f0b640fbc90 msgr2=0x7f0b64104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.909+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 shutdown_connections 2024-03-16T08:14:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:45.909+0000 7f0b6ae44640 1 -- 172.21.15.49:0/3172807156 wait complete. 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 podman[66702]: 2024-03-16 08:14:45.858177985 +0000 UTC m=+0.364208234 container init 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 podman[66702]: 2024-03-16 08:14:45.86085341 +0000 UTC m=+0.366883651 container start 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 ceph-mds[66721]: -- 172.21.15.129:0/1237014281 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639a601ba40 con 0x5639a5fee800 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: starting mds.h at 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 bash[66702]: 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 2024-03-16T08:14:46.111 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:14:45 smithi129 systemd[1]: Started Ceph mds.h for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3172807156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:46.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:46.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:46 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3172807156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:46.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:46 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3172807156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:46.646 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:46 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:47 smithi049 ceph-mon[31942]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:47 smithi049 ceph-mon[31942]: Deploying daemon mds.i on smithi154 2024-03-16T08:14:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:47 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:boot 2024-03-16T08:14:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:47 smithi049 ceph-mon[31942]: fsmap 8 up:standby 2024-03-16T08:14:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:47 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:14:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:47 smithi129 ceph-mon[36561]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:47 smithi129 ceph-mon[36561]: Deploying daemon mds.i on smithi154 2024-03-16T08:14:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:47 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:boot 2024-03-16T08:14:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:47 smithi129 ceph-mon[36561]: fsmap 8 up:standby 2024-03-16T08:14:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:47 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:14:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:47 smithi154 ceph-mon[36471]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:47 smithi154 ceph-mon[36471]: Deploying daemon mds.i on smithi154 2024-03-16T08:14:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:47 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:boot 2024-03-16T08:14:47.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:47 smithi154 ceph-mon[36471]: fsmap 8 up:standby 2024-03-16T08:14:47.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:47 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:14:47.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- 172.21.15.49:0/963267345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 msgr2=0x7eff081004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 --2- 172.21.15.49:0/963267345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff081004f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff04009900 tx=0x7eff0402bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- 172.21.15.49:0/963267345 shutdown_connections 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 --2- 172.21.15.49:0/963267345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 0x7eff08101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 --2- 172.21.15.49:0/963267345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 0x7eff08100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 --2- 172.21.15.49:0/963267345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff081004f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- 172.21.15.49:0/963267345 >> 172.21.15.49:0/963267345 conn(0x7eff080fbc70 msgr2=0x7eff080fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- 172.21.15.49:0/963267345 shutdown_connections 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- 172.21.15.49:0/963267345 wait complete. 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 Processor -- start 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.042+0000 7eff0f780640 1 -- start start 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 0x7eff0819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 0x7eff081a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff0810f660 con 0x7eff081000d0 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff0810f4e0 con 0x7eff081013d0 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff0810f7e0 con 0x7eff08100a30 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:48.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41862/0 (socket says 172.21.15.49:41862) 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 -- 172.21.15.49:0/2769055555 learned_addr learned my addr 172.21.15.49:0/2769055555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0dd1d640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 0x7eff081a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0cd1b640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 0x7eff0819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 -- 172.21.15.49:0/2769055555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 msgr2=0x7eff0819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 0x7eff0819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 -- 172.21.15.49:0/2769055555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 msgr2=0x7eff081a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 0x7eff081a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 -- 172.21.15.49:0/2769055555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081a1940 con 0x7eff081000d0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0d51c640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff04009230 tx=0x7eff0402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04041070 con 0x7eff081000d0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04004450 con 0x7eff081000d0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff081a1bd0 con 0x7eff081000d0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.043+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04003e20 con 0x7eff081000d0 2024-03-16T08:14:48.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.044+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a20c0 con 0x7eff081000d0 2024-03-16T08:14:48.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.044+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0810b000 con 0x7eff081000d0 2024-03-16T08:14:48.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.046+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7eff04044070 con 0x7eff081000d0 2024-03-16T08:14:48.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.046+0000 7efefa7fc640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 0x7efef40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:48.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.046+0000 7eff0cd1b640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 0x7efef40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:48.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.046+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7eff040f0560 con 0x7eff081000d0 2024-03-16T08:14:48.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.047+0000 7eff0cd1b640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 0x7efef40a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efefc009bd0 tx=0x7efefc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:48.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.050+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff040bd450 con 0x7eff081000d0 2024-03-16T08:14:48.225 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 systemd[1]: Starting Ceph mds.i for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:48.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.274+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff0802d370 con 0x7eff081000d0 2024-03-16T08:14:48.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.274+0000 7efefa7fc640 1 -- 172.21.15.49:0/2769055555 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7eff040c14f0 con 0x7eff081000d0 2024-03-16T08:14:48.275 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:48.275 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-16T08:14:48.276 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.276+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 msgr2=0x7efef40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.276+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 0x7efef40a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efefc009bd0 tx=0x7efefc005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.276+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 msgr2=0x7eff08199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff04009230 tx=0x7eff0402c650 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 shutdown_connections 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff081013d0 0x7eff081a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff08100a30 0x7eff0819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efef40a2800 0x7efef40a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 --2- 172.21.15.49:0/2769055555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff081000d0 0x7eff08199d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 >> 172.21.15.49:0/2769055555 conn(0x7eff080fbc70 msgr2=0x7eff080fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 shutdown_connections 2024-03-16T08:14:48.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:48.277+0000 7eff0f780640 1 -- 172.21.15.49:0/2769055555 wait complete. 2024-03-16T08:14:48.597 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 podman[67974]: 2024-03-16 08:14:48.235406788 +0000 UTC m=+0.017845590 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:48.598 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 podman[67974]: 2024-03-16 08:14:48.35285784 +0000 UTC m=+0.135296634 container create fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:14:48.975 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 podman[67974]: 2024-03-16 08:14:48.597982218 +0000 UTC m=+0.380421021 container init fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:14:48.975 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 podman[67974]: 2024-03-16 08:14:48.600877886 +0000 UTC m=+0.383316682 container start fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T08:14:48.975 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 ceph-mds[67993]: -- 172.21.15.154:0/2422271259 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7ccd5ba40 con 0x55f7ccd2ec00 2024-03-16T08:14:48.976 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: starting mds.i at 2024-03-16T08:14:48.976 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 bash[67974]: fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 2024-03-16T08:14:48.976 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:14:48 smithi154 systemd[1]: Started Ceph mds.i for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:49.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:49.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2769055555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:49.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:49.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:49.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:49 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2769055555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:49.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:49.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:49.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:49 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2769055555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:49.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:49 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:50.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:50.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 -- 172.21.15.49:0/1578743039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a41018f0 msgr2=0x7f41a4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 --2- 172.21.15.49:0/1578743039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a41018f0 0x7f41a4101d70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f418c009900 tx=0x7f418c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 -- 172.21.15.49:0/1578743039 shutdown_connections 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 --2- 172.21.15.49:0/1578743039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 0x7f41a4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 --2- 172.21.15.49:0/1578743039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a41018f0 0x7f41a4101d70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 --2- 172.21.15.49:0/1578743039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a4100700 0x7f41a4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 -- 172.21.15.49:0/1578743039 >> 172.21.15.49:0/1578743039 conn(0x7f41a40fbc70 msgr2=0x7f41a40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 -- 172.21.15.49:0/1578743039 shutdown_connections 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.413+0000 7f41a9f77640 1 -- 172.21.15.49:0/1578743039 wait complete. 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 Processor -- start 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 -- start start 2024-03-16T08:14:50.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a41018f0 0x7f41a4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 0x7f41a419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410c3b0 con 0x7f41a4100700 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410c230 con 0x7f41a41022b0 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a9f77640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410c530 con 0x7f41a41018f0 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41888/0 (socket says 172.21.15.49:41888) 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 -- 172.21.15.49:0/2175360591 learned_addr learned my addr 172.21.15.49:0/2175360591 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a2ffd640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a41018f0 0x7f41a4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a3fff640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 0x7f41a419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 -- 172.21.15.49:0/2175360591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a41018f0 msgr2=0x7f41a4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a41018f0 0x7f41a4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 -- 172.21.15.49:0/2175360591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 msgr2=0x7f41a419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:50.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 0x7f41a419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.414+0000 7f41a37fe640 1 -- 172.21.15.49:0/2175360591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a419f730 con 0x7f41a4100700 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a37fe640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f419400e9c0 tx=0x7f419400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419400cd80 con 0x7f41a4100700 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41940035f0 con 0x7f41a4100700 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a419fa20 con 0x7f41a4100700 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194010690 con 0x7f41a4100700 2024-03-16T08:14:50.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.415+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a419ff40 con 0x7f41a4100700 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.416+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a40685d0 con 0x7f41a4100700 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.417+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f41940037a0 con 0x7f41a4100700 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.418+0000 7f41a0ff9640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 0x7f41900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.418+0000 7f41a2ffd640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 0x7f41900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.418+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f41940ce970 con 0x7f41a4100700 2024-03-16T08:14:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.418+0000 7f41a2ffd640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 0x7f41900a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f418c02c240 tx=0x7f418c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:50.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.423+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f419409beb0 con 0x7f41a4100700 2024-03-16T08:14:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:50 smithi049 ceph-mon[31942]: Deploying daemon mds.g on smithi049 2024-03-16T08:14:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:50 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:boot 2024-03-16T08:14:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:50 smithi049 ceph-mon[31942]: fsmap 9 up:standby 2024-03-16T08:14:50.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:50 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:14:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:50 smithi129 ceph-mon[36561]: Deploying daemon mds.g on smithi049 2024-03-16T08:14:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:50 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:boot 2024-03-16T08:14:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:50 smithi129 ceph-mon[36561]: fsmap 9 up:standby 2024-03-16T08:14:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:50 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:14:50.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.655+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41a41a0170 con 0x7f41a4100700 2024-03-16T08:14:50.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.656+0000 7f41a0ff9640 1 -- 172.21.15.49:0/2175360591 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f419409b7b0 con 0x7f41a4100700 2024-03-16T08:14:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:50.657 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T08:14:50.658 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2024-03-16T08:14:50.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 msgr2=0x7f41900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:50.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 0x7f41900a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f418c02c240 tx=0x7f418c02df90 comp rx=0 tx=0).stop 2024-03-16T08:14:50.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 msgr2=0x7f41a4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f419400e9c0 tx=0x7f419400ee80 comp rx=0 tx=0).stop 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 shutdown_connections 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41a41022b0 0x7f41a419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a41018f0 0x7f41a4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f41900a2800 0x7f41900a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 --2- 172.21.15.49:0/2175360591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41a4100700 0x7f41a4197b10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 >> 172.21.15.49:0/2175360591 conn(0x7f41a40fbc70 msgr2=0x7f41a40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 shutdown_connections 2024-03-16T08:14:50.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:50.659+0000 7f41a9f77640 1 -- 172.21.15.49:0/2175360591 wait complete. 2024-03-16T08:14:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:50 smithi154 ceph-mon[36471]: Deploying daemon mds.g on smithi049 2024-03-16T08:14:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:50 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:boot 2024-03-16T08:14:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:50 smithi154 ceph-mon[36471]: fsmap 9 up:standby 2024-03-16T08:14:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:50 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:14:51.265 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 podman[72964]: 2024-03-16 08:14:51.190415977 +0000 UTC m=+0.125150007 container create 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-16T08:14:51.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:51 smithi049 ceph-mon[31942]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:51.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:51 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2175360591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:51.538 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 podman[72964]: 2024-03-16 08:14:51.462416894 +0000 UTC m=+0.397150939 container init 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-16T08:14:51.538 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 podman[72964]: 2024-03-16 08:14:51.465332008 +0000 UTC m=+0.400066047 container start 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T08:14:51.538 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 ceph-mds[72983]: -- 172.21.15.49:0/3966793995 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633af431a40 con 0x5633af404800 2024-03-16T08:14:51.538 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g[72979]: starting mds.g at 2024-03-16T08:14:51.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:51 smithi129 ceph-mon[36561]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:51.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:51 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2175360591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:51 smithi154 ceph-mon[36471]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:51 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2175360591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:52.037 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 bash[72964]: 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a 2024-03-16T08:14:52.037 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:14:51 smithi049 systemd[1]: Started Ceph mds.g for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:52.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 -- 172.21.15.49:0/2175395797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 msgr2=0x7f4b7c100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 --2- 172.21.15.49:0/2175395797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c100e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009900 tx=0x7f4b7002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 -- 172.21.15.49:0/2175395797 shutdown_connections 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 --2- 172.21.15.49:0/2175395797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1013d0 0x7f4b7c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 --2- 172.21.15.49:0/2175395797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c100e90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 --2- 172.21.15.49:0/2175395797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1000d0 0x7f4b7c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 -- 172.21.15.49:0/2175395797 >> 172.21.15.49:0/2175395797 conn(0x7f4b7c0fbc70 msgr2=0x7f4b7c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 -- 172.21.15.49:0/2175395797 shutdown_connections 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.785+0000 7f4b82281640 1 -- 172.21.15.49:0/2175395797 wait complete. 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 Processor -- start 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 -- start start 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1000d0 0x7f4b7c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1013d0 0x7f4b7c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10f670 con 0x7f4b7c100a30 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10f4f0 con 0x7f4b7c1013d0 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b82281640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10f7f0 con 0x7f4b7c1000d0 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.786+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49684/0 (socket says 172.21.15.49:49684) 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 -- 172.21.15.49:0/960853701 learned_addr learned my addr 172.21.15.49:0/960853701 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b8081e640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1013d0 0x7f4b7c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7bfff640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1000d0 0x7f4b7c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 -- 172.21.15.49:0/960853701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1000d0 msgr2=0x7f4b7c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1000d0 0x7f4b7c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 -- 172.21.15.49:0/960853701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1013d0 msgr2=0x7f4b7c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1013d0 0x7f4b7c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 -- 172.21.15.49:0/960853701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c1a1940 con 0x7f4b7c100a30 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b7b7fe640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009230 tx=0x7f4b70005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70041070 con 0x7f4b7c100a30 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c1a1bd0 con 0x7f4b7c100a30 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b700054e0 con 0x7f4b7c100a30 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70034440 con 0x7f4b7c100a30 2024-03-16T08:14:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.787+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c1a20c0 con 0x7f4b7c100a30 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.788+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7c10b000 con 0x7f4b7c100a30 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.790+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4b70044070 con 0x7f4b7c100a30 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.791+0000 7f4b797fa640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 0x7f4b640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.791+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4b700f0640 con 0x7f4b7c100a30 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.791+0000 7f4b7bfff640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 0x7f4b640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.792+0000 7f4b7bfff640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 0x7f4b640a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c19b200 tx=0x7f4b6c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:52.795+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4b700bd530 con 0x7f4b7c100a30 2024-03-16T08:14:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: Deploying daemon mds.a on smithi049 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.49:6838/2351115602,v1:172.21.15.49:6839/2351115602] up:boot 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: fsmap 10 up:standby 2024-03-16T08:14:52.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:52 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:14:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: Deploying daemon mds.a on smithi049 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.49:6838/2351115602,v1:172.21.15.49:6839/2351115602] up:boot 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: fsmap 10 up:standby 2024-03-16T08:14:52.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:52 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:14:52.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:52.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: Deploying daemon mds.a on smithi049 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.49:6838/2351115602,v1:172.21.15.49:6839/2351115602] up:boot 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: fsmap 10 up:standby 2024-03-16T08:14:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:52 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:14:53.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.022+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b7c02d1b0 con 0x7f4b7c100a30 2024-03-16T08:14:53.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.022+0000 7f4b797fa640 1 -- 172.21.15.49:0/960853701 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f4b700c15d0 con 0x7f4b7c100a30 2024-03-16T08:14:53.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:53.023 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T08:14:53.024 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 11 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.024+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 msgr2=0x7f4b640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.024+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 0x7f4b640a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c19b200 tx=0x7f4b6c00b390 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.024+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 msgr2=0x7f4b7c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.024+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009230 tx=0x7f4b70005dd0 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 shutdown_connections 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b7c1013d0 0x7f4b7c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4b640a2800 0x7f4b640a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b7c100a30 0x7f4b7c19a270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 --2- 172.21.15.49:0/960853701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b7c1000d0 0x7f4b7c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 >> 172.21.15.49:0/960853701 conn(0x7f4b7c0fbc70 msgr2=0x7f4b7c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 shutdown_connections 2024-03-16T08:14:53.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:53.025+0000 7f4b82281640 1 -- 172.21.15.49:0/960853701 wait complete. 2024-03-16T08:14:53.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:53 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/960853701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:53.788 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:53 smithi049 systemd[1]: Starting Ceph mds.a for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:53.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:53 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/960853701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:54.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:53 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/960853701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:54.193 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:53 smithi049 podman[73327]: 2024-03-16 08:14:53.844104326 +0000 UTC m=+0.018624653 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:54.193 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:53 smithi049 podman[73327]: 2024-03-16 08:14:53.957364814 +0000 UTC m=+0.131885141 container create 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T08:14:54.193 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 podman[73327]: 2024-03-16 08:14:54.193639893 +0000 UTC m=+0.368160219 container init 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T08:14:54.537 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 podman[73327]: 2024-03-16 08:14:54.19646482 +0000 UTC m=+0.370985146 container start 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T08:14:54.537 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 ceph-mds[73347]: -- 172.21.15.49:0/1595853326 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d648a1a40 con 0x563d64874800 2024-03-16T08:14:54.538 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a[73343]: starting mds.a at 2024-03-16T08:14:54.538 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 bash[73327]: 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 2024-03-16T08:14:54.538 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:14:54 smithi049 systemd[1]: Started Ceph mds.a for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:55.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- 172.21.15.49:0/3777854918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 msgr2=0x7f8180103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 --2- 172.21.15.49:0/3777854918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f8180103240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8170009900 tx=0x7f817002bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- 172.21.15.49:0/3777854918 shutdown_connections 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 --2- 172.21.15.49:0/3777854918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 0x7f8180109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 --2- 172.21.15.49:0/3777854918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 0x7f81800ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 --2- 172.21.15.49:0/3777854918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f8180103240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- 172.21.15.49:0/3777854918 >> 172.21.15.49:0/3777854918 conn(0x7f81800fb410 msgr2=0x7f81800fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- 172.21.15.49:0/3777854918 shutdown_connections 2024-03-16T08:14:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- 172.21.15.49:0/3777854918 wait complete. 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 Processor -- start 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.162+0000 7f818715b640 1 -- start start 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 0x7f8180102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 0x7f8180100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f818010d4a0 con 0x7f8180102e20 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f818010d320 con 0x7f81800ff890 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f818010d620 con 0x7f8180100250 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49730/0 (socket says 172.21.15.49:49730) 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 -- 172.21.15.49:0/2549924243 learned_addr learned my addr 172.21.15.49:0/2549924243 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f8184ef7640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 0x7f8180102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f817ffff640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 0x7f8180100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 -- 172.21.15.49:0/2549924243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 msgr2=0x7f8180100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 0x7f8180100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 -- 172.21.15.49:0/2549924243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 msgr2=0x7f8180102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 0x7f8180102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 -- 172.21.15.49:0/2549924243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8180101930 con 0x7f8180102e20 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f81856f8640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f817800eaa0 tx=0x7f817800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817800cdb0 con 0x7f8180102e20 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8178005020 con 0x7f8180102e20 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.163+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81801a1c30 con 0x7f8180102e20 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.164+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81780116b0 con 0x7f8180102e20 2024-03-16T08:14:55.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.164+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81801a21d0 con 0x7f8180102e20 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.165+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8180108d30 con 0x7f8180102e20 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.166+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8178005560 con 0x7f8180102e20 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.166+0000 7f817dffb640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 0x7f81680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.166+0000 7f8184ef7640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 0x7f81680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.166+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81780ce7d0 con 0x7f8180102e20 2024-03-16T08:14:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.167+0000 7f8184ef7640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 0x7f81680a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8170000c00 tx=0x7f8170005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.171+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8178016030 con 0x7f8180102e20 2024-03-16T08:14:55.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.395+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818002d360 con 0x7f8180102e20 2024-03-16T08:14:55.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.395+0000 7f817dffb640 1 -- 172.21.15.49:0/2549924243 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f817809bd10 con 0x7f8180102e20 2024-03-16T08:14:55.396 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:55.397 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T08:14:55.397 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2024-03-16T08:14:55.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 msgr2=0x7f81680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:55.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 0x7f81680a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8170000c00 tx=0x7f8170005fd0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 msgr2=0x7f81801010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f817800eaa0 tx=0x7f817800ef60 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 shutdown_connections 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f81680a2880 0x7f81680a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8180102e20 0x7f81801010a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8180100250 0x7f8180100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 --2- 172.21.15.49:0/2549924243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81800ff890 0x7f8180102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 >> 172.21.15.49:0/2549924243 conn(0x7f81800fb410 msgr2=0x7f8180105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 shutdown_connections 2024-03-16T08:14:55.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:55.397+0000 7f818715b640 1 -- 172.21.15.49:0/2549924243 wait complete. 2024-03-16T08:14:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: Deploying daemon mds.b on smithi129 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.49:6840/1594522521,v1:172.21.15.49:6841/1594522521] up:boot 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: fsmap 11 up:standby 2024-03-16T08:14:55.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:55 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:14:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: Deploying daemon mds.b on smithi129 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.49:6840/1594522521,v1:172.21.15.49:6841/1594522521] up:boot 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: fsmap 11 up:standby 2024-03-16T08:14:55.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:55 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:14:55.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:55.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: Deploying daemon mds.b on smithi129 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.49:6840/1594522521,v1:172.21.15.49:6841/1594522521] up:boot 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: fsmap 11 up:standby 2024-03-16T08:14:55.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:55 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:14:56.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:56 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2549924243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:56.611 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:56 smithi129 systemd[1]: Starting Ceph mds.b for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:14:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:56 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2549924243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:56 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2549924243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:57.005 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:56 smithi129 podman[67022]: 2024-03-16 08:14:56.644998189 +0000 UTC m=+0.018169233 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:14:57.005 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:56 smithi129 podman[67022]: 2024-03-16 08:14:56.743498987 +0000 UTC m=+0.116670067 container create 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T08:14:57.005 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:56 smithi129 podman[67022]: 2024-03-16 08:14:56.979415928 +0000 UTC m=+0.352586973 container init 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T08:14:57.005 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:56 smithi129 podman[67022]: 2024-03-16 08:14:56.982259425 +0000 UTC m=+0.355430455 container start 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-16T08:14:57.341 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mds[67042]: -- 172.21.15.129:0/1862526586 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55768137fa40 con 0x557681352c00 2024-03-16T08:14:57.342 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: starting mds.b at 2024-03-16T08:14:57.342 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:57 smithi129 bash[67022]: 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 2024-03-16T08:14:57.342 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:14:57 smithi129 systemd[1]: Started Ceph mds.b for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:14:57.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:14:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 -- 172.21.15.49:0/950932262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 msgr2=0x7fa7a8104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 --2- 172.21.15.49:0/950932262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8104710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009900 tx=0x7fa7a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 -- 172.21.15.49:0/950932262 shutdown_connections 2024-03-16T08:14:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 --2- 172.21.15.49:0/950932262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8107580 0x7fa7a8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 --2- 172.21.15.49:0/950932262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8104c50 0x7fa7a8107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 --2- 172.21.15.49:0/950932262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8104710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.522+0000 7fa7b0036640 1 -- 172.21.15.49:0/950932262 >> 172.21.15.49:0/950932262 conn(0x7fa7a80fbc70 msgr2=0x7fa7a80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- 172.21.15.49:0/950932262 shutdown_connections 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- 172.21.15.49:0/950932262 wait complete. 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 Processor -- start 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- start start 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 0x7fa7a81002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8107580 0x7fa7a8100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810d4a0 con 0x7fa7a8102320 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810d320 con 0x7fa7a8104c50 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.523+0000 7fa7b0036640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810d620 con 0x7fa7a8107580 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49758/0 (socket says 172.21.15.49:49758) 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 -- 172.21.15.49:0/4037445192 learned_addr learned my addr 172.21.15.49:0/4037445192 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7ad5d1640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 0x7fa7a81002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7ae5d3640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8107580 0x7fa7a8100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 -- 172.21.15.49:0/4037445192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8107580 msgr2=0x7fa7a8100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8107580 0x7fa7a8100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 -- 172.21.15.49:0/4037445192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 msgr2=0x7fa7a81002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 0x7fa7a81002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 -- 172.21.15.49:0/4037445192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a81010a0 con 0x7fa7a8102320 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7ad5d1640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 0x7fa7a81002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7addd2640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009230 tx=0x7fa7a402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4041070 con 0x7fa7a8102320 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a81a1c00 con 0x7fa7a8102320 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a4004450 con 0x7fa7a8102320 2024-03-16T08:14:57.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003e20 con 0x7fa7a8102320 2024-03-16T08:14:57.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.524+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a21a0 con 0x7fa7a8102320 2024-03-16T08:14:57.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.525+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a80fd760 con 0x7fa7a8102320 2024-03-16T08:14:57.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.527+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa7a4044070 con 0x7fa7a8102320 2024-03-16T08:14:57.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.527+0000 7fa79effd640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 0x7fa7940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.527+0000 7fa7ad5d1640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 0x7fa7940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.527+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa7a40f04f0 con 0x7fa7a8102320 2024-03-16T08:14:57.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.528+0000 7fa7ad5d1640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 0x7fa7940a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8101a00 tx=0x7fa798005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:57.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.535+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7a40bd3e0 con 0x7fa7a8102320 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:57 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:57 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.759+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7a802d1b0 con 0x7fa7a8102320 2024-03-16T08:14:57.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.760+0000 7fa79effd640 1 -- 172.21.15.49:0/4037445192 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fa7a40c1480 con 0x7fa7a8102320 2024-03-16T08:14:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:57.762 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-16T08:14:57.762 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 13 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 msgr2=0x7fa7940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 0x7fa7940a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8101a00 tx=0x7fa798005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 msgr2=0x7fa7a8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009230 tx=0x7fa7a402c650 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 shutdown_connections 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a8107580 0x7fa7a8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7a8104c50 0x7fa7a81002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa7940a2800 0x7fa7940a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 --2- 172.21.15.49:0/4037445192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8102320 0x7fa7a8101c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 >> 172.21.15.49:0/4037445192 conn(0x7fa7a80fbc70 msgr2=0x7fa7a8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.762+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 shutdown_connections 2024-03-16T08:14:57.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.763+0000 7fa7b0036640 1 -- 172.21.15.49:0/4037445192 wait complete. 2024-03-16T08:14:57.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:57.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.774 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.774 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:57 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:57.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:14:57.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 -- 172.21.15.49:0/3465065932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c100700 msgr2=0x7fb98c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 --2- 172.21.15.49:0/3465065932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c100700 0x7fb98c100b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb988009860 tx=0x7fb98802bb70 comp rx=0 tx=0).stop 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 -- 172.21.15.49:0/3465065932 shutdown_connections 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 --2- 172.21.15.49:0/3465065932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c1022b0 0x7fb98c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 --2- 172.21.15.49:0/3465065932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1018f0 0x7fb98c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 --2- 172.21.15.49:0/3465065932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c100700 0x7fb98c100b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 -- 172.21.15.49:0/3465065932 >> 172.21.15.49:0/3465065932 conn(0x7fb98c0fbc70 msgr2=0x7fb98c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 -- 172.21.15.49:0/3465065932 shutdown_connections 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.889+0000 7fb993eff640 1 -- 172.21.15.49:0/3465065932 wait complete. 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 Processor -- start 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 -- start start 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 0x7fb98c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1022b0 0x7fb98c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10c350 con 0x7fb98c100700 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10c1d0 con 0x7fb98c1018f0 2024-03-16T08:14:57.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb993eff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10c4d0 con 0x7fb98c1022b0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb991c9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb991c9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49776/0 (socket says 172.21.15.49:49776) 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb99149a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 0x7fb98c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb991c9b640 1 -- 172.21.15.49:0/3670215201 learned_addr learned my addr 172.21.15.49:0/3670215201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.890+0000 7fb99249c640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1022b0 0x7fb98c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 -- 172.21.15.49:0/3670215201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1022b0 msgr2=0x7fb98c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1022b0 0x7fb98c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 -- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 msgr2=0x7fb98c196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 -- 172.21.15.49:0/3670215201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19e670 con 0x7fb98c1018f0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb991c9b640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb99149a640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 0x7fb98c196f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb97c005fd0 tx=0x7fb97c00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00ccc0 con 0x7fb98c1018f0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c0035f0 con 0x7fb98c1018f0 2024-03-16T08:14:57.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00ce60 con 0x7fb98c1018f0 2024-03-16T08:14:57.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c0689a0 con 0x7fb98c1018f0 2024-03-16T08:14:57.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.891+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c068f40 con 0x7fb98c1018f0 2024-03-16T08:14:57.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.892+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c107530 con 0x7fb98c1018f0 2024-03-16T08:14:57.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.893+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb97c0037a0 con 0x7fb98c1018f0 2024-03-16T08:14:57.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.894+0000 7fb982ffd640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 0x7fb9780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:57.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.894+0000 7fb991c9b640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 0x7fb9780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:57.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.894+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb97c0ce700 con 0x7fb98c1018f0 2024-03-16T08:14:57.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.894+0000 7fb991c9b640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 0x7fb9780a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb9880057c0 tx=0x7fb988034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:57.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:57.897+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb97c009510 con 0x7fb98c1018f0 2024-03-16T08:14:58.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.089+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98c107950 con 0x7fb98c1018f0 2024-03-16T08:14:58.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.090+0000 7fb982ffd640 1 -- 172.21.15.49:0/3670215201 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18421 (secure 0 0 0) 0x7fb97c09bc40 con 0x7fb98c1018f0 2024-03-16T08:14:58.091 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:58.091 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:14:13.878528+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 msgr2=0x7fb9780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 0x7fb9780a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb9880057c0 tx=0x7fb988034ab0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 msgr2=0x7fb98c196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 0x7fb98c196f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb97c005fd0 tx=0x7fb97c00ed90 comp rx=0 tx=0).stop 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 shutdown_connections 2024-03-16T08:14:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb98c1022b0 0x7fb98c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb98c1018f0 0x7fb98c196f90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb9780a2880 0x7fb9780a4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 --2- 172.21.15.49:0/3670215201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c100700 0x7fb98c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 >> 172.21.15.49:0/3670215201 conn(0x7fb98c0fbc70 msgr2=0x7fb98c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 shutdown_connections 2024-03-16T08:14:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.092+0000 7fb993eff640 1 -- 172.21.15.49:0/3670215201 wait complete. 2024-03-16T08:14:58.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:14:58.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:14:58.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 -- 172.21.15.49:0/2434305782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff81018d0 msgr2=0x7efff8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 --2- 172.21.15.49:0/2434305782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff81018d0 0x7efff8101d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effec009900 tx=0x7effec02bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:58.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 -- 172.21.15.49:0/2434305782 shutdown_connections 2024-03-16T08:14:58.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 --2- 172.21.15.49:0/2434305782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff8102290 0x7efff8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 --2- 172.21.15.49:0/2434305782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff81018d0 0x7efff8101d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 --2- 172.21.15.49:0/2434305782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81006e0 0x7efff8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 -- 172.21.15.49:0/2434305782 >> 172.21.15.49:0/2434305782 conn(0x7efff80fbc70 msgr2=0x7efff80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 -- 172.21.15.49:0/2434305782 shutdown_connections 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.217+0000 7efffe310640 1 -- 172.21.15.49:0/2434305782 wait complete. 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 Processor -- start 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 -- start start 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff81006e0 0x7efff8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81018d0 0x7efff8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c360 con 0x7efff8102290 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c1e0 con 0x7efff81006e0 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffe310640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c4e0 con 0x7efff81018d0 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49796/0 (socket says 172.21.15.49:49796) 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 -- 172.21.15.49:0/2567768511 learned_addr learned my addr 172.21.15.49:0/2567768511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efff7fff640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff81006e0 0x7efff8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efff77fe640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81018d0 0x7efff8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 -- 172.21.15.49:0/2567768511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81018d0 msgr2=0x7efff8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81018d0 0x7efff8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 -- 172.21.15.49:0/2567768511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff81006e0 msgr2=0x7efff8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff81006e0 0x7efff8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.218+0000 7efffc8ad640 1 -- 172.21.15.49:0/2567768511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff819f770 con 0x7efff8102290 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efffc8ad640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efff000eb10 tx=0x7efff000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0004b20 con 0x7efff8102290 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff819fa60 con 0x7efff8102290 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff0004cc0 con 0x7efff8102290 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0010760 con 0x7efff8102290 2024-03-16T08:14:58.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.219+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff819ff80 con 0x7efff8102290 2024-03-16T08:14:58.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.220+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff81a0150 con 0x7efff8102290 2024-03-16T08:14:58.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.221+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efff0010900 con 0x7efff8102290 2024-03-16T08:14:58.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.222+0000 7efff57fa640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 0x7effe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.222+0000 7efff7fff640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 0x7effe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.222+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efff00ced60 con 0x7efff8102290 2024-03-16T08:14:58.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.222+0000 7efff7fff640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 0x7effe00a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff8101730 tx=0x7effe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.226+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efff0053740 con 0x7efff8102290 2024-03-16T08:14:58.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.426+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efff81a0320 con 0x7efff8102290 2024-03-16T08:14:58.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.427+0000 7efff57fa640 1 -- 172.21.15.49:0/2567768511 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7efff009c2a0 con 0x7efff8102290 2024-03-16T08:14:58.428 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:14:58.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 msgr2=0x7effe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 0x7effe00a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff8101730 tx=0x7effe8005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:58.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 msgr2=0x7efff819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efff000eb10 tx=0x7efff000c2d0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 shutdown_connections 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7effe00a2800 0x7effe00a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efff8102290 0x7efff819f100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efff81018d0 0x7efff8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 --2- 172.21.15.49:0/2567768511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efff81006e0 0x7efff8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 >> 172.21.15.49:0/2567768511 conn(0x7efff80fbc70 msgr2=0x7efff80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 shutdown_connections 2024-03-16T08:14:58.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.429+0000 7efffe310640 1 -- 172.21.15.49:0/2567768511 wait complete. 2024-03-16T08:14:58.441 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:boot 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: fsmap 12 up:standby 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4037445192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:58 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3670215201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 -- 172.21.15.49:0/3982429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c107560 msgr2=0x7ffa2c1099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 --2- 172.21.15.49:0/3982429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c107560 0x7ffa2c1099b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009ca0 tx=0x7ffa2402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 -- 172.21.15.49:0/3982429197 shutdown_connections 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 --2- 172.21.15.49:0/3982429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c107560 0x7ffa2c1099b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 --2- 172.21.15.49:0/3982429197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c104c30 0x7ffa2c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 --2- 172.21.15.49:0/3982429197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c102300 0x7ffa2c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 -- 172.21.15.49:0/3982429197 >> 172.21.15.49:0/3982429197 conn(0x7ffa2c0fbd10 msgr2=0x7ffa2c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 -- 172.21.15.49:0/3982429197 shutdown_connections 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.549+0000 7ffa32405640 1 -- 172.21.15.49:0/3982429197 wait complete. 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 Processor -- start 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 -- start start 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c102300 0x7ffa2c101c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c107560 0x7ffa2c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c10d4b0 con 0x7ffa2c104c30 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c10d330 con 0x7ffa2c107560 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa32405640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c10d630 con 0x7ffa2c102300 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49810/0 (socket says 172.21.15.49:49810) 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 -- 172.21.15.49:0/2638946447 learned_addr learned my addr 172.21.15.49:0/2638946447 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa309a2640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c107560 0x7ffa2c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2bfff640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c102300 0x7ffa2c101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 -- 172.21.15.49:0/2638946447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c102300 msgr2=0x7ffa2c101c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c102300 0x7ffa2c101c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 -- 172.21.15.49:0/2638946447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c107560 msgr2=0x7ffa2c1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c107560 0x7ffa2c1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.550+0000 7ffa2b7fe640 1 -- 172.21.15.49:0/2638946447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c101010 con 0x7ffa2c104c30 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa2b7fe640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000e9f0 tx=0x7ffa2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000ccc0 con 0x7ffa2c104c30 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20005500 con 0x7ffa2c104c30 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20010690 con 0x7ffa2c104c30 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c1a1c10 con 0x7ffa2c104c30 2024-03-16T08:14:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.551+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c1a2110 con 0x7ffa2c104c30 2024-03-16T08:14:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.552+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2c0fd800 con 0x7ffa2c104c30 2024-03-16T08:14:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.553+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ffa200046c0 con 0x7ffa2c104c30 2024-03-16T08:14:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.553+0000 7ffa297fa640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 0x7ffa140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.554+0000 7ffa2bfff640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 0x7ffa140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.554+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffa200ce900 con 0x7ffa2c104c30 2024-03-16T08:14:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.554+0000 7ffa2bfff640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 0x7ffa140a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009bd0 tx=0x7ffa1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.558+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffa2009be40 con 0x7ffa2c104c30 2024-03-16T08:14:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:boot 2024-03-16T08:14:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: fsmap 12 up:standby 2024-03-16T08:14:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:14:58.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4037445192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:58 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3670215201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:boot 2024-03-16T08:14:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: fsmap 12 up:standby 2024-03-16T08:14:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:14:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4037445192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:58 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3670215201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:14:58.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.793+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa2c02d3b0 con 0x7ffa2c104c30 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.794+0000 7ffa297fa640 1 -- 172.21.15.49:0/2638946447 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7ffa2009b740 con 0x7ffa2c104c30 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:14:58.795 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 msgr2=0x7ffa140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 0x7ffa140a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009bd0 tx=0x7ffa1c005f90 comp rx=0 tx=0).stop 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 msgr2=0x7ffa2c1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000e9f0 tx=0x7ffa2000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 shutdown_connections 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa2c107560 0x7ffa2c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa140a2800 0x7ffa140a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa2c104c30 0x7ffa2c1002b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 --2- 172.21.15.49:0/2638946447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa2c102300 0x7ffa2c101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 >> 172.21.15.49:0/2638946447 conn(0x7ffa2c0fbd10 msgr2=0x7ffa2c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 shutdown_connections 2024-03-16T08:14:58.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.797+0000 7ffa32405640 1 -- 172.21.15.49:0/2638946447 wait complete. 2024-03-16T08:14:58.814 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T08:14:58.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T08:14:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 -- 172.21.15.49:0/792675410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801029e0 msgr2=0x7feb80102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 --2- 172.21.15.49:0/792675410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801029e0 0x7feb80102e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb74009900 tx=0x7feb7402bc30 comp rx=0 tx=0).stop 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 -- 172.21.15.49:0/792675410 shutdown_connections 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 --2- 172.21.15.49:0/792675410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801033a0 0x7feb80109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 --2- 172.21.15.49:0/792675410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801029e0 0x7feb80102e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 --2- 172.21.15.49:0/792675410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 -- 172.21.15.49:0/792675410 >> 172.21.15.49:0/792675410 conn(0x7feb800fcd40 msgr2=0x7feb800ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 -- 172.21.15.49:0/792675410 shutdown_connections 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.942+0000 7feb860b5640 1 -- 172.21.15.49:0/792675410 wait complete. 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 Processor -- start 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 -- start start 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801029e0 0x7feb8019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d450 con 0x7feb801033a0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d2d0 con 0x7feb801017f0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb860b5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d5d0 con 0x7feb801029e0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49824/0 (socket says 172.21.15.49:49824) 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:37694/0 (socket says 172.21.15.49:37694) 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7ffff640 1 -- 172.21.15.49:0/3894514732 learned_addr learned my addr 172.21.15.49:0/3894514732 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.943+0000 7feb7effd640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801029e0 0x7feb8019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 -- 172.21.15.49:0/3894514732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801029e0 msgr2=0x7feb8019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801029e0 0x7feb8019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 -- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 msgr2=0x7feb801a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 -- 172.21.15.49:0/3894514732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801a1940 con 0x7feb801017f0 2024-03-16T08:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7ffff640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7f7fe640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e9c0 tx=0x7feb7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ccc0 con 0x7feb801017f0 2024-03-16T08:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801a1c30 con 0x7feb801017f0 2024-03-16T08:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb700035f0 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ce60 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.944+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801a2180 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.945+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb801a2350 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.946+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7feb700037a0 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.947+0000 7feb7cff9640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 0x7feb680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.947+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feb700ce9f0 con 0x7feb801017f0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.947+0000 7feb7effd640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 0x7feb680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.948+0000 7feb7effd640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 0x7feb680a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb7402c7b0 tx=0x7feb74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:14:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:58.951+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb7009bf30 con 0x7feb801017f0 2024-03-16T08:14:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:14:59.164+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feb801a26e0 con 0x7feb801017f0 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2567768511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2638946447' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:14:59 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:14:59.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2567768511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2638946447' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:14:59.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:14:59 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:00.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2567768511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:15:00.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2638946447' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:14:59 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:00.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.153+0000 7feb7cff9640 1 -- 172.21.15.49:0/3894514732 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7feb7009b830 con 0x7feb801017f0 2024-03-16T08:15:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-03-16T08:15:00.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 msgr2=0x7feb680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 0x7feb680a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb7402c7b0 tx=0x7feb74005fd0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 msgr2=0x7feb80199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e9c0 tx=0x7feb7000ee80 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 shutdown_connections 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb680a2800 0x7feb680a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb801033a0 0x7feb801a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb801029e0 0x7feb8019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 --2- 172.21.15.49:0/3894514732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb801017f0 0x7feb80199d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 >> 172.21.15.49:0/3894514732 conn(0x7feb800fcd40 msgr2=0x7feb80073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 shutdown_connections 2024-03-16T08:15:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.156+0000 7feb860b5640 1 -- 172.21.15.49:0/3894514732 wait complete. 2024-03-16T08:15:00.168 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T08:15:00.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 -- 172.21.15.49:0/4228209104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c1074c0 msgr2=0x7f600c109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 --2- 172.21.15.49:0/4228209104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c1074c0 0x7f600c109910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6004009ca0 tx=0x7f600402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 -- 172.21.15.49:0/4228209104 shutdown_connections 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 --2- 172.21.15.49:0/4228209104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c1074c0 0x7f600c109910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 --2- 172.21.15.49:0/4228209104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c104b90 0x7f600c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 --2- 172.21.15.49:0/4228209104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 0x7f600c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 -- 172.21.15.49:0/4228209104 >> 172.21.15.49:0/4228209104 conn(0x7f600c0fbc70 msgr2=0x7f600c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 -- 172.21.15.49:0/4228209104 shutdown_connections 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.295+0000 7f60108c0640 1 -- 172.21.15.49:0/4228209104 wait complete. 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 Processor -- start 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 -- start start 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 0x7f600c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c1074c0 0x7f600c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f600c10d460 con 0x7f600c104b90 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f600c10d2e0 con 0x7f600c1074c0 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f60108c0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f600c10d5e0 con 0x7f600c102260 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f6003fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f6003fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49856/0 (socket says 172.21.15.49:49856) 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.296+0000 7f6003fff640 1 -- 172.21.15.49:0/4167753782 learned_addr learned my addr 172.21.15.49:0/4167753782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f600a59c640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 0x7f600c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f600ad9d640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c1074c0 0x7f600c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 -- 172.21.15.49:0/4167753782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 msgr2=0x7f600c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 0x7f600c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 -- 172.21.15.49:0/4167753782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c1074c0 msgr2=0x7f600c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c1074c0 0x7f600c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 -- 172.21.15.49:0/4167753782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c1a1940 con 0x7f600c104b90 2024-03-16T08:15:00.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f6003fff640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9c0 tx=0x7f5ff400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:00.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400cd80 con 0x7f600c104b90 2024-03-16T08:15:00.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c1a1c30 con 0x7f600c104b90 2024-03-16T08:15:00.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff4005500 con 0x7f600c104b90 2024-03-16T08:15:00.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4010690 con 0x7f600c104b90 2024-03-16T08:15:00.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.297+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1a21d0 con 0x7f600c104b90 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.298+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600c108fb0 con 0x7f600c104b90 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.300+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f5ff40046c0 con 0x7f600c104b90 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.300+0000 7f60037fe640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 0x7f5ff80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.300+0000 7f600a59c640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 0x7f5ff80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.300+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f5ff40ced30 con 0x7f600c104b90 2024-03-16T08:15:00.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.301+0000 7f600a59c640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 0x7f5ff80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f600c19b200 tx=0x7f5ffc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:00.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.304+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5ff4053740 con 0x7f600c104b90 2024-03-16T08:15:00.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:00.500+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f600c02d1b0 con 0x7f600c104b90 2024-03-16T08:15:01.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.147+0000 7f60037fe640 1 -- 172.21.15.49:0/4167753782 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f5ff409c100 con 0x7f600c104b90 2024-03-16T08:15:01.148 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-03-16T08:15:01.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 msgr2=0x7f5ff80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:01.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 0x7f5ff80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f600c19b200 tx=0x7f5ffc005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:01.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 msgr2=0x7f600c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:01.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9c0 tx=0x7f5ff400ee80 comp rx=0 tx=0).stop 2024-03-16T08:15:01.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 shutdown_connections 2024-03-16T08:15:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600c1074c0 0x7f600c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5ff80a2800 0x7f5ff80a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f600c104b90 0x7f600c19a270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 --2- 172.21.15.49:0/4167753782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f600c102260 0x7f600c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.150+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 >> 172.21.15.49:0/4167753782 conn(0x7f600c0fbc70 msgr2=0x7f600c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:01.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.153+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 shutdown_connections 2024-03-16T08:15:01.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.153+0000 7f60108c0640 1 -- 172.21.15.49:0/4167753782 wait complete. 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3894514732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T08:15:01.181 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T08:15:01.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T08:15:01.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3894514732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T08:15:01.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.301+0000 7f957f6f8640 1 -- 172.21.15.49:0/2350971065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95781018f0 msgr2=0x7f9578101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- 172.21.15.49:0/2350971065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95781018f0 0x7f9578101d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f956c009860 tx=0x7f956c02bb70 comp rx=0 tx=0).stop 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- 172.21.15.49:0/2350971065 shutdown_connections 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- 172.21.15.49:0/2350971065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781022b0 0x7f9578108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- 172.21.15.49:0/2350971065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95781018f0 0x7f9578101d70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- 172.21.15.49:0/2350971065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9578100700 0x7f9578100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- 172.21.15.49:0/2350971065 >> 172.21.15.49:0/2350971065 conn(0x7f95780fbc90 msgr2=0x7f95780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- 172.21.15.49:0/2350971065 shutdown_connections 2024-03-16T08:15:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- 172.21.15.49:0/2350971065 wait complete. 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 Processor -- start 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- start start 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 0x7f9578198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f957810c380 con 0x7f95781022b0 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f957810c200 con 0x7f95781018f0 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.302+0000 7f957f6f8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f957810c500 con 0x7f9578100700 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957d494640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49882/0 (socket says 172.21.15.49:49882) 2024-03-16T08:15:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957d494640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:54194/0 (socket says 172.21.15.49:54194) 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 -- 172.21.15.49:0/1088545441 learned_addr learned my addr 172.21.15.49:0/1088545441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957cc93640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 0x7f9578198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 -- 172.21.15.49:0/1088545441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 msgr2=0x7f9578197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 -- 172.21.15.49:0/1088545441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 msgr2=0x7f9578198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 0x7f9578198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 -- 172.21.15.49:0/1088545441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819f720 con 0x7f95781022b0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957d494640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957cc93640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 0x7f9578198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957dc95640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f957000eb10 tx=0x7f957000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000ce20 con 0x7f95781022b0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819fa10 con 0x7f95781022b0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570005020 con 0x7f95781022b0 2024-03-16T08:15:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570010690 con 0x7f95781022b0 2024-03-16T08:15:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.303+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957819ff80 con 0x7f95781022b0 2024-03-16T08:15:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.305+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95781a02a0 con 0x7f95781022b0 2024-03-16T08:15:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.306+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9570005540 con 0x7f95781022b0 2024-03-16T08:15:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.307+0000 7f956a7fc640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 0x7f95640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.307+0000 7f957d494640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 0x7f95640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.307+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f95700cf5d0 con 0x7f95781022b0 2024-03-16T08:15:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.307+0000 7f957d494640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 0x7f95640a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9578101750 tx=0x7f9574009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:01.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.311+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f957009c820 con 0x7f95781022b0 2024-03-16T08:15:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3894514732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T08:15:01.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T08:15:01.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:01.566+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f95781a0470 con 0x7f95781022b0 2024-03-16T08:15:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T08:15:02.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:02 smithi154 ceph-mon[36471]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T08:15:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.530+0000 7f956a7fc640 1 -- 172.21.15.49:0/1088545441 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f957009c120 con 0x7f95781022b0 2024-03-16T08:15:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T08:15:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 msgr2=0x7f95640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 0x7f95640a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9578101750 tx=0x7f9574009260 comp rx=0 tx=0).stop 2024-03-16T08:15:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 msgr2=0x7f957819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f957000eb10 tx=0x7f957000ef10 comp rx=0 tx=0).stop 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 shutdown_connections 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f95640a2800 0x7f95640a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95781022b0 0x7f957819f0b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95781018f0 0x7f9578198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 --2- 172.21.15.49:0/1088545441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9578100700 0x7f9578197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 >> 172.21.15.49:0/1088545441 conn(0x7f95780fbc90 msgr2=0x7f9578069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 shutdown_connections 2024-03-16T08:15:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.533+0000 7f957f6f8640 1 -- 172.21.15.49:0/1088545441 wait complete. 2024-03-16T08:15:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T08:15:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T08:15:02.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:02 smithi049 ceph-mon[31942]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T08:15:02.551 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:15:02.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4167753782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:15:02.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T08:15:02.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:02 smithi129 ceph-mon[36561]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 -- 172.21.15.49:0/2582615025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1013b0 msgr2=0x7ff37c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 --2- 172.21.15.49:0/2582615025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1013b0 0x7ff37c108c50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff374009900 tx=0x7ff37402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 -- 172.21.15.49:0/2582615025 shutdown_connections 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 --2- 172.21.15.49:0/2582615025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1013b0 0x7ff37c108c50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 --2- 172.21.15.49:0/2582615025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c1009f0 0x7ff37c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 --2- 172.21.15.49:0/2582615025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c101cc0 0x7ff37c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 -- 172.21.15.49:0/2582615025 >> 172.21.15.49:0/2582615025 conn(0x7ff37c0fb420 msgr2=0x7ff37c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 -- 172.21.15.49:0/2582615025 shutdown_connections 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.664+0000 7ff383797640 1 -- 172.21.15.49:0/2582615025 wait complete. 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 Processor -- start 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 -- start start 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c1013b0 0x7ff37c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 0x7ff37c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c10d4e0 con 0x7ff37c1009f0 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c10d360 con 0x7ff37c101cc0 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff383797640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c10d660 con 0x7ff37c1013b0 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff381533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff381533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41388/0 (socket says 172.21.15.49:41388) 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff381533640 1 -- 172.21.15.49:0/1678519481 learned_addr learned my addr 172.21.15.49:0/1678519481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.665+0000 7ff380d32640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c1013b0 0x7ff37c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381d34640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 0x7ff37c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 -- 172.21.15.49:0/1678519481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c1013b0 msgr2=0x7ff37c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c1013b0 0x7ff37c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 -- 172.21.15.49:0/1678519481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 msgr2=0x7ff37c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 0x7ff37c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 -- 172.21.15.49:0/1678519481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c19f6f0 con 0x7ff37c1009f0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381d34640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 0x7ff37c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff381533640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eaa0 tx=0x7ff37800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378016070 con 0x7ff37c1009f0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c19f9e0 con 0x7ff37c1009f0 2024-03-16T08:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378003e50 con 0x7ff37c1009f0 2024-03-16T08:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780116d0 con 0x7ff37c1009f0 2024-03-16T08:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.666+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c19fee0 con 0x7ff37c1009f0 2024-03-16T08:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.667+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c107b30 con 0x7ff37c1009f0 2024-03-16T08:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.668+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff378019070 con 0x7ff37c1009f0 2024-03-16T08:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.669+0000 7ff3727fc640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.669+0000 7ff380d32640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.669+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff3780ce9f0 con 0x7ff37c1009f0 2024-03-16T08:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.669+0000 7ff380d32640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 0x7ff3680a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005d00 tx=0x7ff36c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:02.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.673+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff37809bbe0 con 0x7ff37c1009f0 2024-03-16T08:15:02.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.858+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff37c02d1b0 con 0x7ff37c1009f0 2024-03-16T08:15:02.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.860+0000 7ff3727fc640 1 -- 172.21.15.49:0/1678519481 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22197 (secure 0 0 0) 0x7ff37809b4e0 con 0x7ff37c1009f0 2024-03-16T08:15:02.861 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:02.861 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":85,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:15:01.568626+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:15:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 msgr2=0x7ff3680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 0x7ff3680a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005d00 tx=0x7ff36c005c50 comp rx=0 tx=0).stop 2024-03-16T08:15:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 msgr2=0x7ff37c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eaa0 tx=0x7ff37800ef60 comp rx=0 tx=0).stop 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 shutdown_connections 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff37c101cc0 0x7ff37c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff37c1013b0 0x7ff37c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 --2- 172.21.15.49:0/1678519481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff37c1009f0 0x7ff37c197ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 >> 172.21.15.49:0/1678519481 conn(0x7ff37c0fb420 msgr2=0x7ff37c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 shutdown_connections 2024-03-16T08:15:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.862+0000 7ff383797640 1 -- 172.21.15.49:0/1678519481 wait complete. 2024-03-16T08:15:02.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:15:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 -- 172.21.15.49:0/2492351332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b4100700 msgr2=0x7f61b4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 --2- 172.21.15.49:0/2492351332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b4100700 0x7f61b4100b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009900 tx=0x7f61a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 -- 172.21.15.49:0/2492351332 shutdown_connections 2024-03-16T08:15:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 --2- 172.21.15.49:0/2492351332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b41022b0 0x7f61b4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 --2- 172.21.15.49:0/2492351332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41018f0 0x7f61b4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 --2- 172.21.15.49:0/2492351332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b4100700 0x7f61b4100b20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 -- 172.21.15.49:0/2492351332 >> 172.21.15.49:0/2492351332 conn(0x7f61b40fbc70 msgr2=0x7f61b40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 -- 172.21.15.49:0/2492351332 shutdown_connections 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.981+0000 7f61b9f59640 1 -- 172.21.15.49:0/2492351332 wait complete. 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 Processor -- start 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 -- start start 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b4100700 0x7f61b4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41022b0 0x7f61b419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61b410d4a0 con 0x7f61b41018f0 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61b410d320 con 0x7f61b4100700 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b9f59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61b410d620 con 0x7f61b41022b0 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41392/0 (socket says 172.21.15.49:41392) 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 -- 172.21.15.49:0/3614272983 learned_addr learned my addr 172.21.15.49:0/3614272983 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b37fe640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b4100700 0x7f61b4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b3fff640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41022b0 0x7f61b419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 -- 172.21.15.49:0/3614272983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41022b0 msgr2=0x7f61b419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41022b0 0x7f61b419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 -- 172.21.15.49:0/3614272983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b4100700 msgr2=0x7f61b4197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b4100700 0x7f61b4197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.982+0000 7f61b2ffd640 1 -- 172.21.15.49:0/3614272983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b419f770 con 0x7f61b41018f0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b2ffd640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e9f0 tx=0x7f61a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a800cd80 con 0x7f61b41018f0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a8005500 con 0x7f61b41018f0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b419fa60 con 0x7f61b41018f0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8010690 con 0x7f61b41018f0 2024-03-16T08:15:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.983+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b419ff60 con 0x7f61b41018f0 2024-03-16T08:15:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.984+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4107a70 con 0x7f61b41018f0 2024-03-16T08:15:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.985+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f61a80046c0 con 0x7f61b41018f0 2024-03-16T08:15:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.986+0000 7f61b0ff9640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 0x7f619c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.986+0000 7f61b37fe640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 0x7f619c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.986+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f61a80ced00 con 0x7f61b41018f0 2024-03-16T08:15:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.987+0000 7f61b37fe640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 0x7f619c0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000c00 tx=0x7f61a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:02.991+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f61a809bef0 con 0x7f61b41018f0 2024-03-16T08:15:03.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.180+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61b402d390 con 0x7f61b41018f0 2024-03-16T08:15:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.181+0000 7f61b0ff9640 1 -- 172.21.15.49:0/3614272983 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22184 (secure 0 0 0) 0x7f61a809b7f0 con 0x7f61b41018f0 2024-03-16T08:15:03.183 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:03.183 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:15:03.149983+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 msgr2=0x7f619c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 0x7f619c0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000c00 tx=0x7f61a4005fd0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 msgr2=0x7f61b4198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e9f0 tx=0x7f61a800edf0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 shutdown_connections 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61b41022b0 0x7f61b419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f619c0a2800 0x7f619c0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b41018f0 0x7f61b4198090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 --2- 172.21.15.49:0/3614272983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61b4100700 0x7f61b4197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 >> 172.21.15.49:0/3614272983 conn(0x7f61b40fbc70 msgr2=0x7f61b4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 shutdown_connections 2024-03-16T08:15:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.184+0000 7f61b9f59640 1 -- 172.21.15.49:0/3614272983 wait complete. 2024-03-16T08:15:03.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- 172.21.15.49:0/268050964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 msgr2=0x7f5324108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 --2- 172.21.15.49:0/268050964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f5324108b20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f531c009ca0 tx=0x7f531c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- 172.21.15.49:0/268050964 shutdown_connections 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 --2- 172.21.15.49:0/268050964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f5324108b20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 --2- 172.21.15.49:0/268050964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 --2- 172.21.15.49:0/268050964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 0x7f5324100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- 172.21.15.49:0/268050964 >> 172.21.15.49:0/268050964 conn(0x7f53240fbc70 msgr2=0x7f53240fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- 172.21.15.49:0/268050964 shutdown_connections 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- 172.21.15.49:0/268050964 wait complete. 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 Processor -- start 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.311+0000 7f5329e85640 1 -- start start 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 0x7f5324104670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f532410c340 con 0x7f5324102290 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f532410c1c0 con 0x7f53241006e0 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f532410c4c0 con 0x7f53241018d0 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41402/0 (socket says 172.21.15.49:41402) 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 -- 172.21.15.49:0/418845357 learned_addr learned my addr 172.21.15.49:0/418845357 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f53237fe640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 0x7f5324104670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5322ffd640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 -- 172.21.15.49:0/418845357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 msgr2=0x7f5324102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 -- 172.21.15.49:0/418845357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 msgr2=0x7f5324104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 0x7f5324104670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 -- 172.21.15.49:0/418845357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5324103a80 con 0x7f5324102290 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5322ffd640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5323fff640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f531c005e30 tx=0x7f531c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c041020 con 0x7f5324102290 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53241a1b40 con 0x7f5324102290 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c034cd0 con 0x7f5324102290 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c039040 con 0x7f5324102290 2024-03-16T08:15:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.312+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a20e0 con 0x7f5324102290 2024-03-16T08:15:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.314+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53241a22b0 con 0x7f5324102290 2024-03-16T08:15:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.315+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f531c044020 con 0x7f5324102290 2024-03-16T08:15:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.315+0000 7f5320ff9640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 0x7f530c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.316+0000 7f53237fe640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 0x7f530c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.316+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f531c0f0d50 con 0x7f5324102290 2024-03-16T08:15:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.316+0000 7f53237fe640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 0x7f530c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5324101730 tx=0x7f531400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.319+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f531c04b020 con 0x7f5324102290 2024-03-16T08:15:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: pgmap v283: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T08:15:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T08:15:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T08:15:03.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T08:15:03.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T08:15:03.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1678519481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: pgmap v283: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T08:15:03.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T08:15:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T08:15:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T08:15:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T08:15:03.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1678519481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:03.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:03.549+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53241a2480 con 0x7f5324102290 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: pgmap v283: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1088545441' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T08:15:03.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1678519481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T08:15:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3614272983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:active 2024-03-16T08:15:04.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T08:15:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T08:15:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3614272983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:04.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:active 2024-03-16T08:15:04.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T08:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.547+0000 7f5320ff9640 1 -- 172.21.15.49:0/418845357 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f531c0bdf40 con 0x7f5324102290 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 msgr2=0x7f530c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 0x7f530c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5324101730 tx=0x7f531400b390 comp rx=0 tx=0).stop 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 msgr2=0x7f53241031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f531c005e30 tx=0x7f531c032ec0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 shutdown_connections 2024-03-16T08:15:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f530c0a2800 0x7f530c0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5324102290 0x7f53241031f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53241018d0 0x7f5324102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 --2- 172.21.15.49:0/418845357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53241006e0 0x7f5324104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 >> 172.21.15.49:0/418845357 conn(0x7f53240fbc70 msgr2=0x7f53240fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 shutdown_connections 2024-03-16T08:15:04.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.550+0000 7f5329e85640 1 -- 172.21.15.49:0/418845357 wait complete. 2024-03-16T08:15:04.563 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-16T08:15:04.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3614272983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] up:active 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T08:15:04.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 -- 172.21.15.49:0/4019204887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8944101800 msgr2=0x7f8944101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:04.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 --2- 172.21.15.49:0/4019204887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8944101800 0x7f8944101c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8934009900 tx=0x7f893402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 -- 172.21.15.49:0/4019204887 shutdown_connections 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 --2- 172.21.15.49:0/4019204887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89441033b0 0x7f8944109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 --2- 172.21.15.49:0/4019204887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441029f0 0x7f8944102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 --2- 172.21.15.49:0/4019204887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8944101800 0x7f8944101c20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 -- 172.21.15.49:0/4019204887 >> 172.21.15.49:0/4019204887 conn(0x7f89440fcd70 msgr2=0x7f89440ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 -- 172.21.15.49:0/4019204887 shutdown_connections 2024-03-16T08:15:04.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.681+0000 7f8948f45640 1 -- 172.21.15.49:0/4019204887 wait complete. 2024-03-16T08:15:04.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 Processor -- start 2024-03-16T08:15:04.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 -- start start 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 0x7f8944197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441033b0 0x7f894419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f894410d450 con 0x7f89441029f0 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f894410d2d0 con 0x7f89441033b0 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8948f45640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f894410d5d0 con 0x7f8944101800 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8941d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8941d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41410/0 (socket says 172.21.15.49:41410) 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8941d9b640 1 -- 172.21.15.49:0/639581997 learned_addr learned my addr 172.21.15.49:0/639581997 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8942d9d640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441033b0 0x7f894419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f894259c640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 0x7f8944197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.682+0000 7f8941d9b640 1 -- 172.21.15.49:0/639581997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 msgr2=0x7f8944197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8941d9b640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 0x7f8944197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8941d9b640 1 -- 172.21.15.49:0/639581997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441033b0 msgr2=0x7f894419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8941d9b640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441033b0 0x7f894419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8941d9b640 1 -- 172.21.15.49:0/639581997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894419f6e0 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f894259c640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 0x7f8944197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8941d9b640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f893800e9f0 tx=0x7f893800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800cd80 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894419f9d0 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938010910 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938018990 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.683+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894419ff20 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.684+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8944108bf0 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.685+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8938010ac0 con 0x7f89441029f0 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.686+0000 7f89337fe640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 0x7f892c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:04.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.686+0000 7f894259c640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 0x7f892c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:04.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.686+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f89380ced10 con 0x7f89441029f0 2024-03-16T08:15:04.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.687+0000 7f894259c640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 0x7f892c0a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f893402c7b0 tx=0x7f8934005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:04.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.691+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f893809bf00 con 0x7f89441029f0 2024-03-16T08:15:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:04.916+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f894402d400 con 0x7f89441029f0 2024-03-16T08:15:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: pgmap v286: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T08:15:05.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:15:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T08:15:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:05.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:05 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T08:15:05.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: pgmap v286: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:05.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T08:15:05.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:15:05.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T08:15:05.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:05.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:05 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T08:15:05.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.549+0000 7f89337fe640 1 -- 172.21.15.49:0/639581997 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f893809b800 con 0x7f89441029f0 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 msgr2=0x7f892c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 0x7f892c0a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f893402c7b0 tx=0x7f8934005fd0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 msgr2=0x7f8944198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f893800e9f0 tx=0x7f893800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 shutdown_connections 2024-03-16T08:15:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89441033b0 0x7f894419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f892c0a2800 0x7f892c0a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89441029f0 0x7f8944198000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 --2- 172.21.15.49:0/639581997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8944101800 0x7f8944197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 >> 172.21.15.49:0/639581997 conn(0x7f89440fcd70 msgr2=0x7f8944105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 shutdown_connections 2024-03-16T08:15:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.552+0000 7f8948f45640 1 -- 172.21.15.49:0/639581997 wait complete. 2024-03-16T08:15:05.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-16T08:15:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: pgmap v286: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:15:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T08:15:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:15:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/418845357' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T08:15:05.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:05.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:05 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 -- 172.21.15.49:0/656955253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12441018f0 msgr2=0x7f1244101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 --2- 172.21.15.49:0/656955253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12441018f0 0x7f1244101d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1234009900 tx=0x7f123402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 -- 172.21.15.49:0/656955253 shutdown_connections 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 --2- 172.21.15.49:0/656955253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 0x7f1244108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 --2- 172.21.15.49:0/656955253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12441018f0 0x7f1244101d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 --2- 172.21.15.49:0/656955253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1244100700 0x7f1244100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 -- 172.21.15.49:0/656955253 >> 172.21.15.49:0/656955253 conn(0x7f12440fbc90 msgr2=0x7f12440fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 -- 172.21.15.49:0/656955253 shutdown_connections 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.685+0000 7f124b752640 1 -- 172.21.15.49:0/656955253 wait complete. 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 Processor -- start 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 -- start start 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12441018f0 0x7f124419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 0x7f124419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f124410c360 con 0x7f1244100700 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f124410c1e0 con 0x7f12441022b0 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f124b752640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f124410c4e0 con 0x7f12441018f0 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41420/0 (socket says 172.21.15.49:41420) 2024-03-16T08:15:05.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 -- 172.21.15.49:0/213833388 learned_addr learned my addr 172.21.15.49:0/213833388 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f1249cef640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 0x7f124419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f1248ced640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12441018f0 0x7f124419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 -- 172.21.15.49:0/213833388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12441018f0 msgr2=0x7f124419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12441018f0 0x7f124419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 -- 172.21.15.49:0/213833388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 msgr2=0x7f124419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 0x7f124419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.686+0000 7f12494ee640 1 -- 172.21.15.49:0/213833388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124419f720 con 0x7f1244100700 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f12494ee640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f124000c970 tx=0x7f124000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240007c10 con 0x7f1244100700 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240007db0 con 0x7f1244100700 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124000f940 con 0x7f1244100700 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124419f9b0 con 0x7f1244100700 2024-03-16T08:15:05.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.687+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124419fe80 con 0x7f1244100700 2024-03-16T08:15:05.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.688+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12441079c0 con 0x7f1244100700 2024-03-16T08:15:05.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.689+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f124000fae0 con 0x7f1244100700 2024-03-16T08:15:05.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.689+0000 7f123a7fc640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 0x7f12300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:05.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.690+0000 7f1248ced640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 0x7f12300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:05.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.690+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12400cec90 con 0x7f1244100700 2024-03-16T08:15:05.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.690+0000 7f1248ced640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 0x7f12300a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1234006010 tx=0x7f1234000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:05.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.694+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f124009bf10 con 0x7f1244100700 2024-03-16T08:15:05.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:05.915+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f124402d1b0 con 0x7f1244100700 2024-03-16T08:15:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T08:15:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T08:15:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T08:15:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T08:15:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T08:15:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T08:15:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T08:15:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T08:15:06.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.559+0000 7f123a7fc640 1 -- 172.21.15.49:0/213833388 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f124402d1b0 con 0x7f1244100700 2024-03-16T08:15:06.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 msgr2=0x7f12300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:06.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 0x7f12300a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1234006010 tx=0x7f1234000bc0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 msgr2=0x7f124406a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f124000c970 tx=0x7f124000ce30 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 shutdown_connections 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12441022b0 0x7f124419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12441018f0 0x7f124419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12300a2800 0x7f12300a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 --2- 172.21.15.49:0/213833388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1244100700 0x7f124406a920 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 >> 172.21.15.49:0/213833388 conn(0x7f12440fbc90 msgr2=0x7f1244104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 shutdown_connections 2024-03-16T08:15:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.562+0000 7f124b752640 1 -- 172.21.15.49:0/213833388 wait complete. 2024-03-16T08:15:06.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-16T08:15:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T08:15:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/639581997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T08:15:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T08:15:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T08:15:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T08:15:06.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T08:15:06.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T08:15:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 -- 172.21.15.49:0/1981490520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 msgr2=0x7f12e4101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 --2- 172.21.15.49:0/1981490520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4101c20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009900 tx=0x7f12d402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 -- 172.21.15.49:0/1981490520 shutdown_connections 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 --2- 172.21.15.49:0/1981490520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 0x7f12e4109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 --2- 172.21.15.49:0/1981490520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 0x7f12e4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 --2- 172.21.15.49:0/1981490520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4101c20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 -- 172.21.15.49:0/1981490520 >> 172.21.15.49:0/1981490520 conn(0x7f12e40fcd70 msgr2=0x7f12e40ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 -- 172.21.15.49:0/1981490520 shutdown_connections 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.695+0000 7f12e9190640 1 -- 172.21.15.49:0/1981490520 wait complete. 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 Processor -- start 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 -- start start 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 0x7f12e4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 0x7f12e419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d450 con 0x7f12e4101800 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d2d0 con 0x7f12e41033b0 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.696+0000 7f12e9190640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d5d0 con 0x7f12e41029f0 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41426/0 (socket says 172.21.15.49:41426) 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 -- 172.21.15.49:0/504552806 learned_addr learned my addr 172.21.15.49:0/504552806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e259c640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 0x7f12e4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e359e640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 0x7f12e419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 -- 172.21.15.49:0/504552806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 msgr2=0x7f12e4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 0x7f12e4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 -- 172.21.15.49:0/504552806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 msgr2=0x7f12e419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 0x7f12e419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 -- 172.21.15.49:0/504552806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e419f710 con 0x7f12e4101800 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e2d9d640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4041070 con 0x7f12e4101800 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d4004450 con 0x7f12e4101800 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e419f9a0 con 0x7f12e4101800 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4034440 con 0x7f12e4101800 2024-03-16T08:15:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.697+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e419fe90 con 0x7f12e4101800 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.698+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e4108bf0 con 0x7f12e4101800 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.699+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f12d4044070 con 0x7f12e4101800 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.700+0000 7f12cbfff640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 0x7f12cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.700+0000 7f12e259c640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 0x7f12cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.700+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12d40f03d0 con 0x7f12e4101800 2024-03-16T08:15:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.700+0000 7f12e259c640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 0x7f12cc0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12e4199000 tx=0x7f12d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:06.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.704+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f12d40bd000 con 0x7f12e4101800 2024-03-16T08:15:06.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:06.929+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f12e402d3f0 con 0x7f12e4101800 2024-03-16T08:15:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: pgmap v287: 97 pgs: 17 creating+peering, 71 active+clean, 9 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T08:15:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T08:15:07.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:active 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T08:15:07.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:07 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T08:15:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: pgmap v287: 97 pgs: 17 creating+peering, 71 active+clean, 9 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T08:15:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:active 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T08:15:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T08:15:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.565+0000 7f12cbfff640 1 -- 172.21.15.49:0/504552806 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f12d40c10a0 con 0x7f12e4101800 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 msgr2=0x7f12cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 0x7f12cc0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12e4199000 tx=0x7f12d8005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 msgr2=0x7f12e4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d402c640 comp rx=0 tx=0).stop 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 shutdown_connections 2024-03-16T08:15:07.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12e41033b0 0x7f12e419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e41029f0 0x7f12e4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f12cc0a2800 0x7f12cc0a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 --2- 172.21.15.49:0/504552806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e4101800 0x7f12e4197af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 >> 172.21.15.49:0/504552806 conn(0x7f12e40fcd70 msgr2=0x7f12e4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 shutdown_connections 2024-03-16T08:15:07.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.569+0000 7f12e9190640 1 -- 172.21.15.49:0/504552806 wait complete. 2024-03-16T08:15:07.583 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-16T08:15:07.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-16T08:15:07.610 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: 2024-03-16T08:15:07.356+0000 7fe0a1403700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T08:15:07.610 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: 2024-03-16T08:15:07.356+0000 7fe0a1403700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-16T08:15:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: pgmap v287: 97 pgs: 17 creating+peering, 71 active+clean, 9 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T08:15:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/213833388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] up:active 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T08:15:07.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:07 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T08:15:07.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.698+0000 7fdf1b00c640 1 -- 172.21.15.49:0/477931077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf14102290 msgr2=0x7fdf14108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.698+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/477931077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf14102290 0x7fdf14108b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009ca0 tx=0x7fdf0c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- 172.21.15.49:0/477931077 shutdown_connections 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/477931077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf14102290 0x7fdf14108b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/477931077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 0x7fdf14101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/477931077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf141006e0 0x7fdf14100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- 172.21.15.49:0/477931077 >> 172.21.15.49:0/477931077 conn(0x7fdf140fbc70 msgr2=0x7fdf140fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- 172.21.15.49:0/477931077 shutdown_connections 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- 172.21.15.49:0/477931077 wait complete. 2024-03-16T08:15:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 Processor -- start 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- start start 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 0x7fdf1419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf14102290 0x7fdf1419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410c360 con 0x7fdf141006e0 2024-03-16T08:15:07.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410c1e0 con 0x7fdf14102290 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf1b00c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410c4e0 con 0x7fdf141018d0 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf18da8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf18da8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41446/0 (socket says 172.21.15.49:41446) 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf18da8640 1 -- 172.21.15.49:0/1338218041 learned_addr learned my addr 172.21.15.49:0/1338218041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf0bfff640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 0x7fdf1419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.699+0000 7fdf195a9640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf14102290 0x7fdf1419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 -- 172.21.15.49:0/1338218041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 msgr2=0x7fdf1419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 0x7fdf1419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 -- 172.21.15.49:0/1338218041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf14102290 msgr2=0x7fdf1419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf14102290 0x7fdf1419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 -- 172.21.15.49:0/1338218041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419f700 con 0x7fdf141006e0 2024-03-16T08:15:07.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf18da8640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9c0 tx=0x7fdf1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000cd80 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419f9f0 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf100035f0 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10010690 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.700+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1419ff90 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.701+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf14107a00 con 0x7fdf141006e0 2024-03-16T08:15:07.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.702+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdf100037a0 con 0x7fdf141006e0 2024-03-16T08:15:07.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.703+0000 7fdf09ffb640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 0x7fdefc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:07.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.703+0000 7fdf0bfff640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 0x7fdefc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:07.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.703+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdf100cecc0 con 0x7fdf141006e0 2024-03-16T08:15:07.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.703+0000 7fdf0bfff640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 0x7fdefc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf14069240 tx=0x7fdf04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:07.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.707+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf1009beb0 con 0x7fdf141006e0 2024-03-16T08:15:07.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.916+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdf14061c20 con 0x7fdefc0a2800 2024-03-16T08:15:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.939+0000 7fdf09ffb640 1 -- 172.21.15.49:0/1338218041 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdf14061c20 con 0x7fdefc0a2800 2024-03-16T08:15:07.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 msgr2=0x7fdefc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 0x7fdefc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf14069240 tx=0x7fdf04005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:07.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 msgr2=0x7fdf140694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:07.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9c0 tx=0x7fdf1000ee80 comp rx=0 tx=0).stop 2024-03-16T08:15:07.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 shutdown_connections 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf14102290 0x7fdf1419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf141018d0 0x7fdf1419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdefc0a2800 0x7fdefc0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 --2- 172.21.15.49:0/1338218041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf141006e0 0x7fdf140694d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 >> 172.21.15.49:0/1338218041 conn(0x7fdf140fbc70 msgr2=0x7fdf14104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:07.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 shutdown_connections 2024-03-16T08:15:07.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:07.941+0000 7fdf1b00c640 1 -- 172.21.15.49:0/1338218041 wait complete. 2024-03-16T08:15:07.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-16T08:15:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 -- 172.21.15.49:0/4064739231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 msgr2=0x7fda2c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 --2- 172.21.15.49:0/4064739231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c100b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda28009900 tx=0x7fda2802bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 -- 172.21.15.49:0/4064739231 shutdown_connections 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 --2- 172.21.15.49:0/4064739231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1022b0 0x7fda2c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 --2- 172.21.15.49:0/4064739231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1018f0 0x7fda2c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 --2- 172.21.15.49:0/4064739231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c100b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 -- 172.21.15.49:0/4064739231 >> 172.21.15.49:0/4064739231 conn(0x7fda2c0fbc90 msgr2=0x7fda2c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 -- 172.21.15.49:0/4064739231 shutdown_connections 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.068+0000 7fda33722640 1 -- 172.21.15.49:0/4064739231 wait complete. 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 Processor -- start 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 -- start start 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1018f0 0x7fda2c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1022b0 0x7fda2c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c10c370 con 0x7fda2c100700 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c10c1f0 con 0x7fda2c1018f0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda33722640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c10c4f0 con 0x7fda2c1022b0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41474/0 (socket says 172.21.15.49:41474) 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 -- 172.21.15.49:0/501875559 learned_addr learned my addr 172.21.15.49:0/501875559 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda30cbd640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1018f0 0x7fda2c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda31cbf640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1022b0 0x7fda2c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 -- 172.21.15.49:0/501875559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1022b0 msgr2=0x7fda2c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1022b0 0x7fda2c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 -- 172.21.15.49:0/501875559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1018f0 msgr2=0x7fda2c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1018f0 0x7fda2c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 -- 172.21.15.49:0/501875559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c19f750 con 0x7fda2c100700 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda314be640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda2802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.069+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28041070 con 0x7fda2c100700 2024-03-16T08:15:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.070+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda28004450 con 0x7fda2c100700 2024-03-16T08:15:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.070+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c19f9e0 con 0x7fda2c100700 2024-03-16T08:15:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.070+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28034440 con 0x7fda2c100700 2024-03-16T08:15:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.070+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c19fed0 con 0x7fda2c100700 2024-03-16T08:15:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.070+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c1a00a0 con 0x7fda2c100700 2024-03-16T08:15:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.072+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fda28044070 con 0x7fda2c100700 2024-03-16T08:15:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.072+0000 7fda227fc640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 0x7fda180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.072+0000 7fda30cbd640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 0x7fda180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.072+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fda280f05e0 con 0x7fda2c100700 2024-03-16T08:15:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.073+0000 7fda30cbd640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 0x7fda180a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda2c199040 tx=0x7fda1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.076+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fda280bd210 con 0x7fda2c100700 2024-03-16T08:15:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.286+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda2c0fdea0 con 0x7fda180a2800 2024-03-16T08:15:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:15:07.921+0000 7fa00fe87700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:15:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:15:07.921+0000 7fa00fe87700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:15:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:15:07.921+0000 7fa00fe87700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:15:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:15:07.921+0000 7fa00fe87700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:15:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:15:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:15:07.921+0000 7fa00fe87700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:15:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.309+0000 7fda227fc640 1 -- 172.21.15.49:0/501875559 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda2c0fdea0 con 0x7fda180a2800 2024-03-16T08:15:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 msgr2=0x7fda180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 0x7fda180a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda2c199040 tx=0x7fda1c005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 msgr2=0x7fda2c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda2802c640 comp rx=0 tx=0).stop 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 shutdown_connections 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda2c1022b0 0x7fda2c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda2c1018f0 0x7fda2c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fda180a2800 0x7fda180a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 --2- 172.21.15.49:0/501875559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda2c100700 0x7fda2c197b30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 >> 172.21.15.49:0/501875559 conn(0x7fda2c0fbc90 msgr2=0x7fda2c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 shutdown_connections 2024-03-16T08:15:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.311+0000 7fda33722640 1 -- 172.21.15.49:0/501875559 wait complete. 2024-03-16T08:15:08.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-16T08:15:08.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 -- 172.21.15.49:0/4182665516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 msgr2=0x7f7e00104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/4182665516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00104650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 -- 172.21.15.49:0/4182665516 shutdown_connections 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/4182665516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e001074c0 0x7f7e00109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/4182665516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e00104b90 0x7f7e00106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/4182665516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00104650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 -- 172.21.15.49:0/4182665516 >> 172.21.15.49:0/4182665516 conn(0x7f7e000fbc70 msgr2=0x7f7e000fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 -- 172.21.15.49:0/4182665516 shutdown_connections 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.435+0000 7f7e04d3b640 1 -- 172.21.15.49:0/4182665516 wait complete. 2024-03-16T08:15:08.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 Processor -- start 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 -- start start 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e00104b90 0x7f7e00198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e001074c0 0x7f7e0019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010d480 con 0x7f7e00102260 2024-03-16T08:15:08.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010d300 con 0x7f7e001074c0 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7e04d3b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010d600 con 0x7f7e00104b90 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41488/0 (socket says 172.21.15.49:41488) 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 -- 172.21.15.49:0/3306215072 learned_addr learned my addr 172.21.15.49:0/3306215072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfed9d640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e001074c0 0x7f7e0019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7df5d9b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e00104b90 0x7f7e00198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 -- 172.21.15.49:0/3306215072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e00104b90 msgr2=0x7f7e00198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e00104b90 0x7f7e00198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 -- 172.21.15.49:0/3306215072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e001074c0 msgr2=0x7f7e0019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e001074c0 0x7f7e0019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.436+0000 7f7dfe59c640 1 -- 172.21.15.49:0/3306215072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0019f770 con 0x7f7e00102260 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7dfe59c640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009230 tx=0x7f7df002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0041070 con 0x7f7e00102260 2024-03-16T08:15:08.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0004450 con 0x7f7e00102260 2024-03-16T08:15:08.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0019fa00 con 0x7f7e00102260 2024-03-16T08:15:08.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0003e20 con 0x7f7e00102260 2024-03-16T08:15:08.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.437+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0019ff40 con 0x7f7e00102260 2024-03-16T08:15:08.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.438+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e000fd760 con 0x7f7e00102260 2024-03-16T08:15:08.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.439+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7df0044070 con 0x7f7e00102260 2024-03-16T08:15:08.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.439+0000 7f7df7fff640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 0x7f7dec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.440+0000 7f7df5d9b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 0x7f7dec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.440+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7df00f0420 con 0x7f7e00102260 2024-03-16T08:15:08.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.440+0000 7f7df5d9b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 0x7f7dec0a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e00199060 tx=0x7f7de8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.444+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7df00bcfc0 con 0x7f7e00102260 2024-03-16T08:15:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.587+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7df00035f0 con 0x7f7e00102260 2024-03-16T08:15:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.663+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7e00061cb0 con 0x7f7dec0a2880 2024-03-16T08:15:08.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.667+0000 7f7df7fff640 1 -- 172.21.15.49:0/3306215072 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7e00061cb0 con 0x7f7dec0a2880 2024-03-16T08:15:08.667 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_0/2c8e7f97-8aff-425d-bbc9-b6ea4c91b080 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 msgr2=0x7f7dec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 0x7f7dec0a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e00199060 tx=0x7f7de8005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 msgr2=0x7f7e00197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009230 tx=0x7f7df002c650 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 shutdown_connections 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e001074c0 0x7f7e0019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e00104b90 0x7f7e00198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.668+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7dec0a2880 0x7f7dec0a4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.669+0000 7f7e04d3b640 1 --2- 172.21.15.49:0/3306215072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e00102260 0x7f7e00197b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.669+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 >> 172.21.15.49:0/3306215072 conn(0x7f7e000fbc70 msgr2=0x7f7e00102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.669+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 shutdown_connections 2024-03-16T08:15:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.669+0000 7f7e04d3b640 1 -- 172.21.15.49:0/3306215072 wait complete. 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:active 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 15 op/s 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:08.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:08 smithi049 ceph-mon[31942]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:08.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-16T08:15:08.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 -- 172.21.15.49:0/3898175408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 msgr2=0x7f4e70101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 --2- 172.21.15.49:0/3898175408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70101d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009900 tx=0x7f4e5802bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 -- 172.21.15.49:0/3898175408 shutdown_connections 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 --2- 172.21.15.49:0/3898175408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e701022b0 0x7f4e70108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 --2- 172.21.15.49:0/3898175408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70101d70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 --2- 172.21.15.49:0/3898175408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e70100700 0x7f4e70100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 -- 172.21.15.49:0/3898175408 >> 172.21.15.49:0/3898175408 conn(0x7f4e700fbc90 msgr2=0x7f4e700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 -- 172.21.15.49:0/3898175408 shutdown_connections 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.793+0000 7f4e759c4640 1 -- 172.21.15.49:0/3898175408 wait complete. 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 Processor -- start 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 -- start start 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e70100700 0x7f4e70104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e701022b0 0x7f4e70103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010c360 con 0x7f4e701018f0 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010c1e0 con 0x7f4e70100700 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e759c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010c4e0 con 0x7f4e701022b0 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41518/0 (socket says 172.21.15.49:41518) 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 -- 172.21.15.49:0/4159990803 learned_addr learned my addr 172.21.15.49:0/4159990803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e6effd640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e70100700 0x7f4e70104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e6f7fe640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e701022b0 0x7f4e70103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 -- 172.21.15.49:0/4159990803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e701022b0 msgr2=0x7f4e70103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e701022b0 0x7f4e70103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 -- 172.21.15.49:0/4159990803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e70100700 msgr2=0x7f4e70104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.794+0000 7f4e67fff640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e70100700 0x7f4e70104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:08.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e67fff640 1 -- 172.21.15.49:0/4159990803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70103aa0 con 0x7f4e701018f0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e67fff640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009230 tx=0x7f4e5802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58041070 con 0x7f4e701018f0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701a1be0 con 0x7f4e701018f0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58004450 con 0x7f4e701018f0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58003e20 con 0x7f4e701018f0 2024-03-16T08:15:08.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.795+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a2180 con 0x7f4e701018f0 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.796+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e701a2350 con 0x7f4e701018f0 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.797+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4e58044070 con 0x7f4e701018f0 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.797+0000 7f4e749c2640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 0x7f4e5c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.798+0000 7f4e6effd640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 0x7f4e5c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.798+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4e580f04b0 con 0x7f4e701018f0 2024-03-16T08:15:08.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.798+0000 7f4e6effd640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 0x7f4e5c0a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e70101750 tx=0x7f4e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:08.802+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4e580bd060 con 0x7f4e701018f0 2024-03-16T08:15:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T08:15:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:active 2024-03-16T08:15:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-16T08:15:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:15:08.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 15 op/s 2024-03-16T08:15:08.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:08.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:08 smithi129 ceph-mon[36561]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/504552806' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T08:15:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] up:active 2024-03-16T08:15:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-16T08:15:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:15:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 15 op/s 2024-03-16T08:15:08.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:08.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:08 smithi154 ceph-mon[36471]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.015+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4e700fde80 con 0x7f4e5c0a2890 2024-03-16T08:15:09.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.037+0000 7f4e749c2640 1 -- 172.21.15.49:0/4159990803 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e700fde80 con 0x7f4e5c0a2890 2024-03-16T08:15:09.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 msgr2=0x7f4e5c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 0x7f4e5c0a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e70101750 tx=0x7f4e60005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 msgr2=0x7f4e70102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009230 tx=0x7f4e5802c650 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 shutdown_connections 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e701022b0 0x7f4e70103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4e5c0a2890 0x7f4e5c0a4d50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e701018f0 0x7f4e70102cd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 --2- 172.21.15.49:0/4159990803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e70100700 0x7f4e70104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 >> 172.21.15.49:0/4159990803 conn(0x7f4e700fbc90 msgr2=0x7f4e700fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 shutdown_connections 2024-03-16T08:15:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.039+0000 7f4e759c4640 1 -- 172.21.15.49:0/4159990803 wait complete. 2024-03-16T08:15:09.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-16T08:15:09.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 -- 172.21.15.49:0/334986442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 msgr2=0x7ff0d0103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/334986442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0103e40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009900 tx=0x7ff0c402bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 -- 172.21.15.49:0/334986442 shutdown_connections 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/334986442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 0x7ff0d010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/334986442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 0x7ff0d0105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/334986442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0103e40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 -- 172.21.15.49:0/334986442 >> 172.21.15.49:0/334986442 conn(0x7ff0d00fef90 msgr2=0x7ff0d01013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 -- 172.21.15.49:0/334986442 shutdown_connections 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.163+0000 7ff0d6b90640 1 -- 172.21.15.49:0/334986442 wait complete. 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 Processor -- start 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 -- start start 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 0x7ff0d019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 0x7ff0d01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010f6e0 con 0x7ff0d0103a20 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010f560 con 0x7ff0d01055d0 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d6b90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010f860 con 0x7ff0d0104c10 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d492c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d492c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41522/0 (socket says 172.21.15.49:41522) 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d492c640 1 -- 172.21.15.49:0/1942926991 learned_addr learned my addr 172.21.15.49:0/1942926991 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0cffff640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 0x7ff0d019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.164+0000 7ff0d512d640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 0x7ff0d01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 -- 172.21.15.49:0/1942926991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 msgr2=0x7ff0d019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 0x7ff0d019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 -- 172.21.15.49:0/1942926991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 msgr2=0x7ff0d01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 0x7ff0d01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 -- 172.21.15.49:0/1942926991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a1980 con 0x7ff0d0103a20 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d492c640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4043020 con 0x7ff0d0103a20 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c40054e0 con 0x7ff0d0103a20 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a1c10 con 0x7ff0d0103a20 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003c40 con 0x7ff0d0103a20 2024-03-16T08:15:09.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.165+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a21b0 con 0x7ff0d0103a20 2024-03-16T08:15:09.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.166+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d010ae10 con 0x7ff0d0103a20 2024-03-16T08:15:09.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.168+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff0c4046020 con 0x7ff0d0103a20 2024-03-16T08:15:09.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.168+0000 7ff0cdffb640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 0x7ff0c00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.168+0000 7ff0cffff640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 0x7ff0c00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.169+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff0c40f08c0 con 0x7ff0d0103a20 2024-03-16T08:15:09.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.169+0000 7ff0cffff640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 0x7ff0c00a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff0d019b270 tx=0x7ff0bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.174+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff0c404d020 con 0x7ff0d0103a20 2024-03-16T08:15:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.387+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff0d0061cb0 con 0x7ff0c00a2910 2024-03-16T08:15:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.391+0000 7ff0cdffb640 1 -- 172.21.15.49:0/1942926991 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff0d0061cb0 con 0x7ff0c00a2910 2024-03-16T08:15:09.391 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_1/f9081ef2-12dc-45ae-ba21-10abb3f5d3ca 2024-03-16T08:15:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 msgr2=0x7ff0c00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 0x7ff0c00a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff0d019b270 tx=0x7ff0bc005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 msgr2=0x7ff0d0199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c402c650 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 shutdown_connections 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d01055d0 0x7ff0d01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104c10 0x7ff0d019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff0c00a2910 0x7ff0c00a4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 --2- 172.21.15.49:0/1942926991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0d0103a20 0x7ff0d0199d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 >> 172.21.15.49:0/1942926991 conn(0x7ff0d00fef90 msgr2=0x7ff0d0107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 shutdown_connections 2024-03-16T08:15:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.393+0000 7ff0d6b90640 1 -- 172.21.15.49:0/1942926991 wait complete. 2024-03-16T08:15:09.404 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.511+0000 7f4f44ac0640 1 -- 172.21.15.49:0/2595771630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f40107540 msgr2=0x7f4f40109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.511+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/2595771630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f40107540 0x7f4f40109930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009ca0 tx=0x7f4f3002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- 172.21.15.49:0/2595771630 shutdown_connections 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/2595771630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f40107540 0x7f4f40109930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/2595771630 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40104c10 0x7f4f40107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/2595771630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f401022e0 0x7f4f401046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- 172.21.15.49:0/2595771630 >> 172.21.15.49:0/2595771630 conn(0x7f4f400fbc70 msgr2=0x7f4f400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- 172.21.15.49:0/2595771630 shutdown_connections 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- 172.21.15.49:0/2595771630 wait complete. 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 Processor -- start 2024-03-16T08:15:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- start start 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 0x7f4f40198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40107540 0x7f4f4019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010d4a0 con 0x7f4f401022e0 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010d320 con 0x7f4f40104c10 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.512+0000 7f4f44ac0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010d620 con 0x7f4f40107540 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41538/0 (socket says 172.21.15.49:41538) 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 -- 172.21.15.49:0/638328027 learned_addr learned my addr 172.21.15.49:0/638328027 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3dd9b640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 0x7f4f40198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3ed9d640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40107540 0x7f4f4019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 -- 172.21.15.49:0/638328027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40107540 msgr2=0x7f4f4019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40107540 0x7f4f4019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 -- 172.21.15.49:0/638328027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 msgr2=0x7f4f40198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 0x7f4f40198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 -- 172.21.15.49:0/638328027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019f730 con 0x7f4f401022e0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3dd9b640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 0x7f4f40198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f3e59c640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800e9f0 tx=0x7f4f3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800cd80 con 0x7f4f401022e0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38010910 con 0x7f4f401022e0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4019fa20 con 0x7f4f401022e0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38018990 con 0x7f4f401022e0 2024-03-16T08:15:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.513+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4019ff70 con 0x7f4f401022e0 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.514+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f400685d0 con 0x7f4f401022e0 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.515+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4f38010ac0 con 0x7f4f401022e0 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.516+0000 7f4f2f7fe640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 0x7f4f280a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.516+0000 7f4f3dd9b640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 0x7f4f280a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.516+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4f380ced90 con 0x7f4f401022e0 2024-03-16T08:15:09.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.516+0000 7f4f3dd9b640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 0x7f4f280a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f40199020 tx=0x7f4f34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.520+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f3809b830 con 0x7f4f401022e0 2024-03-16T08:15:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.743+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f4002d250 con 0x7f4f401022e0 2024-03-16T08:15:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.743+0000 7f4f2f7fe640 1 -- 172.21.15.49:0/638328027 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f4f4002d250 con 0x7f4f401022e0 2024-03-16T08:15:09.744 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:09.744 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:15:09.744 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 msgr2=0x7f4f280a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 0x7f4f280a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f40199020 tx=0x7f4f34005f90 comp rx=0 tx=0).stop 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 msgr2=0x7f4f40197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800e9f0 tx=0x7f4f3800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 shutdown_connections 2024-03-16T08:15:09.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40107540 0x7f4f4019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f40104c10 0x7f4f40198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4f280a2890 0x7f4f280a4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 --2- 172.21.15.49:0/638328027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401022e0 0x7f4f40197b10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.745+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 >> 172.21.15.49:0/638328027 conn(0x7f4f400fbc70 msgr2=0x7f4f40102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.746+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 shutdown_connections 2024-03-16T08:15:09.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.746+0000 7f4f44ac0640 1 -- 172.21.15.49:0/638328027 wait complete. 2024-03-16T08:15:09.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:09 smithi049 ceph-mon[31942]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-16T08:15:09.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:09 smithi049 ceph-mon[31942]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:09 smithi049 ceph-mon[31942]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:09 smithi049 ceph-mon[31942]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:15:09.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:09 smithi129 ceph-mon[36561]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-16T08:15:09.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:09 smithi129 ceph-mon[36561]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:09 smithi129 ceph-mon[36561]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:09 smithi129 ceph-mon[36561]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 -- 172.21.15.49:0/882289194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c102a00 msgr2=0x7fe12c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 --2- 172.21.15.49:0/882289194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c102a00 0x7fe12c102e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009900 tx=0x7fe11c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 -- 172.21.15.49:0/882289194 shutdown_connections 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 --2- 172.21.15.49:0/882289194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c1033c0 0x7fe12c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 --2- 172.21.15.49:0/882289194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c102a00 0x7fe12c102e80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 --2- 172.21.15.49:0/882289194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c101810 0x7fe12c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 -- 172.21.15.49:0/882289194 >> 172.21.15.49:0/882289194 conn(0x7fe12c0fcd80 msgr2=0x7fe12c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 -- 172.21.15.49:0/882289194 shutdown_connections 2024-03-16T08:15:09.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.867+0000 7fe1337c6640 1 -- 172.21.15.49:0/882289194 wait complete. 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 Processor -- start 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 -- start start 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c102a00 0x7fe12c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c1033c0 0x7fe12c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c10d470 con 0x7fe12c101810 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c10d2f0 con 0x7fe12c1033c0 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe1337c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c10d5f0 con 0x7fe12c102a00 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41552/0 (socket says 172.21.15.49:41552) 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 -- 172.21.15.49:0/264337341 learned_addr learned my addr 172.21.15.49:0/264337341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe130d61640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c102a00 0x7fe12c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131d63640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c1033c0 0x7fe12c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 -- 172.21.15.49:0/264337341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c102a00 msgr2=0x7fe12c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c102a00 0x7fe12c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 -- 172.21.15.49:0/264337341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c1033c0 msgr2=0x7fe12c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c1033c0 0x7fe12c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.868+0000 7fe131562640 1 -- 172.21.15.49:0/264337341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c19f6e0 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe131562640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe12800e9c0 tx=0x7fe12800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12800cde0 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe128010910 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c19f9d0 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128018a00 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.869+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c19ff40 con 0x7fe12c101810 2024-03-16T08:15:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.870+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c108b30 con 0x7fe12c101810 2024-03-16T08:15:09.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.871+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe12801f020 con 0x7fe12c101810 2024-03-16T08:15:09.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.871+0000 7fe1227fc640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 0x7fe1180a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:09.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.872+0000 7fe130d61640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 0x7fe1180a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:09.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.872+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe1280cf190 con 0x7fe12c101810 2024-03-16T08:15:09.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.872+0000 7fe130d61640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 0x7fe1180a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02c7b0 tx=0x7fe11c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:09.875+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe12809c280 con 0x7fe12c101810 2024-03-16T08:15:09.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:09 smithi154 ceph-mon[36471]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-16T08:15:09.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:09 smithi154 ceph-mon[36471]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:09 smithi154 ceph-mon[36471]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:09.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:09 smithi154 ceph-mon[36471]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:15:10.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.059+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe12c02d460 con 0x7fe12c101810 2024-03-16T08:15:10.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.061+0000 7fe1227fc640 1 -- 172.21.15.49:0/264337341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22184 (secure 0 0 0) 0x7fe12809bb80 con 0x7fe12c101810 2024-03-16T08:15:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:10.061 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:15:03.149983+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 msgr2=0x7fe1180a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 0x7fe1180a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02c7b0 tx=0x7fe11c005fd0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 msgr2=0x7fe12c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe12800e9c0 tx=0x7fe12800ee80 comp rx=0 tx=0).stop 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 shutdown_connections 2024-03-16T08:15:10.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe12c1033c0 0x7fe12c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12c102a00 0x7fe12c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe1180a2910 0x7fe1180a4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 --2- 172.21.15.49:0/264337341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe12c101810 0x7fe12c197ac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 >> 172.21.15.49:0/264337341 conn(0x7fe12c0fcd80 msgr2=0x7fe12c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 shutdown_connections 2024-03-16T08:15:10.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.063+0000 7fe1337c6640 1 -- 172.21.15.49:0/264337341 wait complete. 2024-03-16T08:15:10.074 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T08:15:10.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 -- 172.21.15.49:0/874441913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 msgr2=0x7f922c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 --2- 172.21.15.49:0/874441913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c101460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9228009900 tx=0x7f922802bc30 comp rx=0 tx=0).stop 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 -- 172.21.15.49:0/874441913 shutdown_connections 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 --2- 172.21.15.49:0/874441913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c1082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 --2- 172.21.15.49:0/874441913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c101460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 --2- 172.21.15.49:0/874441913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 -- 172.21.15.49:0/874441913 >> 172.21.15.49:0/874441913 conn(0x7f922c0fb400 msgr2=0x7f922c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 -- 172.21.15.49:0/874441913 shutdown_connections 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.185+0000 7f923444b640 1 -- 172.21.15.49:0/874441913 wait complete. 2024-03-16T08:15:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 Processor -- start 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 -- start start 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10b2e0 con 0x7f922c100fe0 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10b160 con 0x7f922c0ffe90 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f923444b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10b460 con 0x7f922c1019a0 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92321e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92319e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92319e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41580/0 (socket says 172.21.15.49:41580) 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92321e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:53750/0 (socket says 172.21.15.49:53750) 2024-03-16T08:15:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92319e6640 1 -- 172.21.15.49:0/3988205685 learned_addr learned my addr 172.21.15.49:0/3988205685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.186+0000 7f92329e8640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 -- 172.21.15.49:0/3988205685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 msgr2=0x7f922c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 -- 172.21.15.49:0/3988205685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 msgr2=0x7f922c195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 -- 172.21.15.49:0/3988205685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c19d540 con 0x7f922c100fe0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92321e7640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92329e8640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92319e6640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9228000c00 tx=0x7f922802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228041070 con 0x7f922c100fe0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c19d7d0 con 0x7f922c100fe0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9228004450 con 0x7f922c100fe0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228034440 con 0x7f922c100fe0 2024-03-16T08:15:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.187+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c19dcc0 con 0x7f922c100fe0 2024-03-16T08:15:10.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.188+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922c107270 con 0x7f922c100fe0 2024-03-16T08:15:10.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.189+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9228044070 con 0x7f922c100fe0 2024-03-16T08:15:10.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.190+0000 7f92237fe640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 0x7f92180a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.190+0000 7f92321e7640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 0x7f92180a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.190+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f92280f03f0 con 0x7f922c100fe0 2024-03-16T08:15:10.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.190+0000 7f92321e7640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 0x7f92180a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f921c00bf00 tx=0x7f921c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:10.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.194+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f92280bd030 con 0x7f922c100fe0 2024-03-16T08:15:10.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.373+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f922c061c20 con 0x7f92180a2890 2024-03-16T08:15:10.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.381+0000 7f92237fe640 1 -- 172.21.15.49:0/3988205685 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258857 (secure 0 0 0) 0x7f922c061c20 con 0x7f92180a2890 2024-03-16T08:15:10.384 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:10.385 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 msgr2=0x7f92180a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 0x7f92180a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f921c00bf00 tx=0x7f921c00d040 comp rx=0 tx=0).stop 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 msgr2=0x7f922c195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9228000c00 tx=0x7f922802c640 comp rx=0 tx=0).stop 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 shutdown_connections 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c1019a0 0x7f922c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f92180a2890 0x7f92180a4d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f922c100fe0 0x7f922c195e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 --2- 172.21.15.49:0/3988205685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f922c0ffe90 0x7f922c195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 >> 172.21.15.49:0/3988205685 conn(0x7f922c0fb400 msgr2=0x7f922c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:10.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 shutdown_connections 2024-03-16T08:15:10.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.384+0000 7f923444b640 1 -- 172.21.15.49:0/3988205685 wait complete. 2024-03-16T08:15:10.395 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-16T08:15:09.580770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594766,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":281,"num_write_kb":2965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356140,"kb_used_data":11112,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585684,"statfs":{"total":1151940427776,"available":1151575740416,"internally_reserved":0,"allocated":11378688,"data_stored":7088062,"data_compressed":108399,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4398,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001741"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227166+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:15:03.227166+0000","last_peered":"2024-03-16T08:15:03.227166+0000","last_clean":"2024-03-16T08:15:03.227166+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:15:03.227166+0000","last_undegraded":"2024-03-16T08:15:03.227166+0000","last_fullsized":"2024-03-16T08:15:03.227166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226759+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:15:03.226759+0000","last_peered":"2024-03-16T08:15:03.226759+0000","last_clean":"2024-03-16T08:15:03.226759+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:15:03.226759+0000","last_undegraded":"2024-03-16T08:15:03.226759+0000","last_fullsized":"2024-03-16T08:15:03.226759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227962+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:15:03.227962+0000","last_peered":"2024-03-16T08:15:03.227962+0000","last_clean":"2024-03-16T08:15:03.227962+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:15:03.227962+0000","last_undegraded":"2024-03-16T08:15:03.227962+0000","last_fullsized":"2024-03-16T08:15:03.227962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154943+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:15:03.154943+0000","last_peered":"2024-03-16T08:15:03.154943+0000","last_clean":"2024-03-16T08:15:03.154943+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:15:03.154943+0000","last_undegraded":"2024-03-16T08:15:03.154943+0000","last_fullsized":"2024-03-16T08:15:03.154943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243590+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:15:03.243590+0000","last_peered":"2024-03-16T08:15:03.243590+0000","last_clean":"2024-03-16T08:15:03.243590+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:15:03.243590+0000","last_undegraded":"2024-03-16T08:15:03.243590+0000","last_fullsized":"2024-03-16T08:15:03.243590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227148+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:15:03.227148+0000","last_peered":"2024-03-16T08:15:03.227148+0000","last_clean":"2024-03-16T08:15:03.227148+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:15:03.227148+0000","last_undegraded":"2024-03-16T08:15:03.227148+0000","last_fullsized":"2024-03-16T08:15:03.227148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227784+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:15:03.227784+0000","last_peered":"2024-03-16T08:15:03.227784+0000","last_clean":"2024-03-16T08:15:03.227784+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:15:03.227784+0000","last_undegraded":"2024-03-16T08:15:03.227784+0000","last_fullsized":"2024-03-16T08:15:03.227784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153230+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:15:03.153230+0000","last_peered":"2024-03-16T08:15:03.153230+0000","last_clean":"2024-03-16T08:15:03.153230+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:15:03.153230+0000","last_undegraded":"2024-03-16T08:15:03.153230+0000","last_fullsized":"2024-03-16T08:15:03.153230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227764+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:15:03.227764+0000","last_peered":"2024-03-16T08:15:03.227764+0000","last_clean":"2024-03-16T08:15:03.227764+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:15:03.227764+0000","last_undegraded":"2024-03-16T08:15:03.227764+0000","last_fullsized":"2024-03-16T08:15:03.227764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227212+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:15:03.227212+0000","last_peered":"2024-03-16T08:15:03.227212+0000","last_clean":"2024-03-16T08:15:03.227212+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:15:03.227212+0000","last_undegraded":"2024-03-16T08:15:03.227212+0000","last_fullsized":"2024-03-16T08:15:03.227212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228196+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:15:03.228196+0000","last_peered":"2024-03-16T08:15:03.228196+0000","last_clean":"2024-03-16T08:15:03.228196+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:15:03.228196+0000","last_undegraded":"2024-03-16T08:15:03.228196+0000","last_fullsized":"2024-03-16T08:15:03.228196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154911+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:15:03.154911+0000","last_peered":"2024-03-16T08:15:03.154911+0000","last_clean":"2024-03-16T08:15:03.154911+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:15:03.154911+0000","last_undegraded":"2024-03-16T08:15:03.154911+0000","last_fullsized":"2024-03-16T08:15:03.154911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226996+0000","last_change":"2024-03-16T08:15:02.168429+0000","last_active":"2024-03-16T08:15:03.226996+0000","last_peered":"2024-03-16T08:15:03.226996+0000","last_clean":"2024-03-16T08:15:03.226996+0000","last_became_active":"2024-03-16T08:15:02.167835+0000","last_became_peered":"2024-03-16T08:15:02.167835+0000","last_unstale":"2024-03-16T08:15:03.226996+0000","last_undegraded":"2024-03-16T08:15:03.226996+0000","last_fullsized":"2024-03-16T08:15:03.226996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:00:22.118862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228200+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:15:03.228200+0000","last_peered":"2024-03-16T08:15:03.228200+0000","last_clean":"2024-03-16T08:15:03.228200+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:15:03.228200+0000","last_undegraded":"2024-03-16T08:15:03.228200+0000","last_fullsized":"2024-03-16T08:15:03.228200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226497+0000","last_change":"2024-03-16T08:15:02.175165+0000","last_active":"2024-03-16T08:15:03.226497+0000","last_peered":"2024-03-16T08:15:03.226497+0000","last_clean":"2024-03-16T08:15:03.226497+0000","last_became_active":"2024-03-16T08:15:02.173186+0000","last_became_peered":"2024-03-16T08:15:02.173186+0000","last_unstale":"2024-03-16T08:15:03.226497+0000","last_undegraded":"2024-03-16T08:15:03.226497+0000","last_fullsized":"2024-03-16T08:15:03.226497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:28.302644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227637+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:15:03.227637+0000","last_peered":"2024-03-16T08:15:03.227637+0000","last_clean":"2024-03-16T08:15:03.227637+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:15:03.227637+0000","last_undegraded":"2024-03-16T08:15:03.227637+0000","last_fullsized":"2024-03-16T08:15:03.227637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227177+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:15:03.227177+0000","last_peered":"2024-03-16T08:15:03.227177+0000","last_clean":"2024-03-16T08:15:03.227177+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:15:03.227177+0000","last_undegraded":"2024-03-16T08:15:03.227177+0000","last_fullsized":"2024-03-16T08:15:03.227177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226389+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:15:03.226389+0000","last_peered":"2024-03-16T08:15:03.226389+0000","last_clean":"2024-03-16T08:15:03.226389+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:15:03.226389+0000","last_undegraded":"2024-03-16T08:15:03.226389+0000","last_fullsized":"2024-03-16T08:15:03.226389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227336+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:15:03.227336+0000","last_peered":"2024-03-16T08:15:03.227336+0000","last_clean":"2024-03-16T08:15:03.227336+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:15:03.227336+0000","last_undegraded":"2024-03-16T08:15:03.227336+0000","last_fullsized":"2024-03-16T08:15:03.227336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.575498+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:15:06.575498+0000","last_peered":"2024-03-16T08:15:06.575498+0000","last_clean":"2024-03-16T08:15:06.575498+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:15:06.575498+0000","last_undegraded":"2024-03-16T08:15:06.575498+0000","last_fullsized":"2024-03-16T08:15:06.575498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153790+0000","last_change":"2024-03-16T08:15:02.169282+0000","last_active":"2024-03-16T08:15:03.153790+0000","last_peered":"2024-03-16T08:15:03.153790+0000","last_clean":"2024-03-16T08:15:03.153790+0000","last_became_active":"2024-03-16T08:15:02.168163+0000","last_became_peered":"2024-03-16T08:15:02.168163+0000","last_unstale":"2024-03-16T08:15:03.153790+0000","last_undegraded":"2024-03-16T08:15:03.153790+0000","last_fullsized":"2024-03-16T08:15:03.153790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:00:05.780516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154014+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:15:03.154014+0000","last_peered":"2024-03-16T08:15:03.154014+0000","last_clean":"2024-03-16T08:15:03.154014+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:15:03.154014+0000","last_undegraded":"2024-03-16T08:15:03.154014+0000","last_fullsized":"2024-03-16T08:15:03.154014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227452+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:15:03.227452+0000","last_peered":"2024-03-16T08:15:03.227452+0000","last_clean":"2024-03-16T08:15:03.227452+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:15:03.227452+0000","last_undegraded":"2024-03-16T08:15:03.227452+0000","last_fullsized":"2024-03-16T08:15:03.227452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243968+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:15:03.243968+0000","last_peered":"2024-03-16T08:15:03.243968+0000","last_clean":"2024-03-16T08:15:03.243968+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:15:03.243968+0000","last_undegraded":"2024-03-16T08:15:03.243968+0000","last_fullsized":"2024-03-16T08:15:03.243968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227920+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:15:03.227920+0000","last_peered":"2024-03-16T08:15:03.227920+0000","last_clean":"2024-03-16T08:15:03.227920+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:15:03.227920+0000","last_undegraded":"2024-03-16T08:15:03.227920+0000","last_fullsized":"2024-03-16T08:15:03.227920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243690+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:15:03.243690+0000","last_peered":"2024-03-16T08:15:03.243690+0000","last_clean":"2024-03-16T08:15:03.243690+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:15:03.243690+0000","last_undegraded":"2024-03-16T08:15:03.243690+0000","last_fullsized":"2024-03-16T08:15:03.243690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154563+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:15:03.154563+0000","last_peered":"2024-03-16T08:15:03.154563+0000","last_clean":"2024-03-16T08:15:03.154563+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:15:03.154563+0000","last_undegraded":"2024-03-16T08:15:03.154563+0000","last_fullsized":"2024-03-16T08:15:03.154563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.567546+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:15:05.567546+0000","last_peered":"2024-03-16T08:15:05.567546+0000","last_clean":"2024-03-16T08:15:05.567546+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:15:05.567546+0000","last_undegraded":"2024-03-16T08:15:05.567546+0000","last_fullsized":"2024-03-16T08:15:05.567546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243906+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:15:03.243906+0000","last_peered":"2024-03-16T08:15:03.243906+0000","last_clean":"2024-03-16T08:15:03.243906+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:15:03.243906+0000","last_undegraded":"2024-03-16T08:15:03.243906+0000","last_fullsized":"2024-03-16T08:15:03.243906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.574344+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:15:06.574344+0000","last_peered":"2024-03-16T08:15:06.574344+0000","last_clean":"2024-03-16T08:15:06.574344+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:15:06.574344+0000","last_undegraded":"2024-03-16T08:15:06.574344+0000","last_fullsized":"2024-03-16T08:15:06.574344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154514+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:15:03.154514+0000","last_peered":"2024-03-16T08:15:03.154514+0000","last_clean":"2024-03-16T08:15:03.154514+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:15:03.154514+0000","last_undegraded":"2024-03-16T08:15:03.154514+0000","last_fullsized":"2024-03-16T08:15:03.154514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226935+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:15:03.226935+0000","last_peered":"2024-03-16T08:15:03.226935+0000","last_clean":"2024-03-16T08:15:03.226935+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:15:03.226935+0000","last_undegraded":"2024-03-16T08:15:03.226935+0000","last_fullsized":"2024-03-16T08:15:03.226935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153319+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:15:03.153319+0000","last_peered":"2024-03-16T08:15:03.153319+0000","last_clean":"2024-03-16T08:15:03.153319+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:15:03.153319+0000","last_undegraded":"2024-03-16T08:15:03.153319+0000","last_fullsized":"2024-03-16T08:15:03.153319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153942+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:15:03.153942+0000","last_peered":"2024-03-16T08:15:03.153942+0000","last_clean":"2024-03-16T08:15:03.153942+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:15:03.153942+0000","last_undegraded":"2024-03-16T08:15:03.153942+0000","last_fullsized":"2024-03-16T08:15:03.153942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.225995+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:15:03.225995+0000","last_peered":"2024-03-16T08:15:03.225995+0000","last_clean":"2024-03-16T08:15:03.225995+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:15:03.225995+0000","last_undegraded":"2024-03-16T08:15:03.225995+0000","last_fullsized":"2024-03-16T08:15:03.225995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566528+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:15:05.566528+0000","last_peered":"2024-03-16T08:15:05.566528+0000","last_clean":"2024-03-16T08:15:05.566528+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:15:05.566528+0000","last_undegraded":"2024-03-16T08:15:05.566528+0000","last_fullsized":"2024-03-16T08:15:05.566528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227528+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:15:03.227528+0000","last_peered":"2024-03-16T08:15:03.227528+0000","last_clean":"2024-03-16T08:15:03.227528+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:15:03.227528+0000","last_undegraded":"2024-03-16T08:15:03.227528+0000","last_fullsized":"2024-03-16T08:15:03.227528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227303+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:15:03.227303+0000","last_peered":"2024-03-16T08:15:03.227303+0000","last_clean":"2024-03-16T08:15:03.227303+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:15:03.227303+0000","last_undegraded":"2024-03-16T08:15:03.227303+0000","last_fullsized":"2024-03-16T08:15:03.227303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227620+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:15:03.227620+0000","last_peered":"2024-03-16T08:15:03.227620+0000","last_clean":"2024-03-16T08:15:03.227620+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:15:03.227620+0000","last_undegraded":"2024-03-16T08:15:03.227620+0000","last_fullsized":"2024-03-16T08:15:03.227620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154614+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:15:03.154614+0000","last_peered":"2024-03-16T08:15:03.154614+0000","last_clean":"2024-03-16T08:15:03.154614+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:15:03.154614+0000","last_undegraded":"2024-03-16T08:15:03.154614+0000","last_fullsized":"2024-03-16T08:15:03.154614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228077+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:15:03.228077+0000","last_peered":"2024-03-16T08:15:03.228077+0000","last_clean":"2024-03-16T08:15:03.228077+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:15:03.228077+0000","last_undegraded":"2024-03-16T08:15:03.228077+0000","last_fullsized":"2024-03-16T08:15:03.228077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.565481+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:15:05.565481+0000","last_peered":"2024-03-16T08:15:05.565481+0000","last_clean":"2024-03-16T08:15:05.565481+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:15:05.565481+0000","last_undegraded":"2024-03-16T08:15:05.565481+0000","last_fullsized":"2024-03-16T08:15:05.565481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243773+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:15:03.243773+0000","last_peered":"2024-03-16T08:15:03.243773+0000","last_clean":"2024-03-16T08:15:03.243773+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:15:03.243773+0000","last_undegraded":"2024-03-16T08:15:03.243773+0000","last_fullsized":"2024-03-16T08:15:03.243773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.571589+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:15:06.571589+0000","last_peered":"2024-03-16T08:15:06.571589+0000","last_clean":"2024-03-16T08:15:06.571589+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:15:06.571589+0000","last_undegraded":"2024-03-16T08:15:06.571589+0000","last_fullsized":"2024-03-16T08:15:06.571589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227203+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:15:03.227203+0000","last_peered":"2024-03-16T08:15:03.227203+0000","last_clean":"2024-03-16T08:15:03.227203+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:15:03.227203+0000","last_undegraded":"2024-03-16T08:15:03.227203+0000","last_fullsized":"2024-03-16T08:15:03.227203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566759+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:15:05.566759+0000","last_peered":"2024-03-16T08:15:05.566759+0000","last_clean":"2024-03-16T08:15:05.566759+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:15:05.566759+0000","last_undegraded":"2024-03-16T08:15:05.566759+0000","last_fullsized":"2024-03-16T08:15:05.566759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227567+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:15:03.227567+0000","last_peered":"2024-03-16T08:15:03.227567+0000","last_clean":"2024-03-16T08:15:03.227567+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:15:03.227567+0000","last_undegraded":"2024-03-16T08:15:03.227567+0000","last_fullsized":"2024-03-16T08:15:03.227567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154417+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:15:03.154417+0000","last_peered":"2024-03-16T08:15:03.154417+0000","last_clean":"2024-03-16T08:15:03.154417+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:15:03.154417+0000","last_undegraded":"2024-03-16T08:15:03.154417+0000","last_fullsized":"2024-03-16T08:15:03.154417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227040+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:15:03.227040+0000","last_peered":"2024-03-16T08:15:03.227040+0000","last_clean":"2024-03-16T08:15:03.227040+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:15:03.227040+0000","last_undegraded":"2024-03-16T08:15:03.227040+0000","last_fullsized":"2024-03-16T08:15:03.227040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566316+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:15:05.566316+0000","last_peered":"2024-03-16T08:15:05.566316+0000","last_clean":"2024-03-16T08:15:05.566316+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:15:05.566316+0000","last_undegraded":"2024-03-16T08:15:05.566316+0000","last_fullsized":"2024-03-16T08:15:05.566316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154239+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:15:03.154239+0000","last_peered":"2024-03-16T08:15:03.154239+0000","last_clean":"2024-03-16T08:15:03.154239+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:15:03.154239+0000","last_undegraded":"2024-03-16T08:15:03.154239+0000","last_fullsized":"2024-03-16T08:15:03.154239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243685+0000","last_change":"2024-03-16T08:15:01.171487+0000","last_active":"2024-03-16T08:15:03.243685+0000","last_peered":"2024-03-16T08:15:03.243685+0000","last_clean":"2024-03-16T08:15:03.243685+0000","last_became_active":"2024-03-16T08:15:01.171164+0000","last_became_peered":"2024-03-16T08:15:01.171164+0000","last_unstale":"2024-03-16T08:15:03.243685+0000","last_undegraded":"2024-03-16T08:15:03.243685+0000","last_fullsized":"2024-03-16T08:15:03.243685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:49:52.984972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227049+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:15:03.227049+0000","last_peered":"2024-03-16T08:15:03.227049+0000","last_clean":"2024-03-16T08:15:03.227049+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:15:03.227049+0000","last_undegraded":"2024-03-16T08:15:03.227049+0000","last_fullsized":"2024-03-16T08:15:03.227049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226025+0000","last_change":"2024-03-16T08:15:01.168022+0000","last_active":"2024-03-16T08:15:03.226025+0000","last_peered":"2024-03-16T08:15:03.226025+0000","last_clean":"2024-03-16T08:15:03.226025+0000","last_became_active":"2024-03-16T08:15:01.167316+0000","last_became_peered":"2024-03-16T08:15:01.167316+0000","last_unstale":"2024-03-16T08:15:03.226025+0000","last_undegraded":"2024-03-16T08:15:03.226025+0000","last_fullsized":"2024-03-16T08:15:03.226025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:08:56.392344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227292+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:15:03.227292+0000","last_peered":"2024-03-16T08:15:03.227292+0000","last_clean":"2024-03-16T08:15:03.227292+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:15:03.227292+0000","last_undegraded":"2024-03-16T08:15:03.227292+0000","last_fullsized":"2024-03-16T08:15:03.227292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226814+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:15:03.226814+0000","last_peered":"2024-03-16T08:15:03.226814+0000","last_clean":"2024-03-16T08:15:03.226814+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:15:03.226814+0000","last_undegraded":"2024-03-16T08:15:03.226814+0000","last_fullsized":"2024-03-16T08:15:03.226814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228175+0000","last_change":"2024-03-16T08:15:02.174717+0000","last_active":"2024-03-16T08:15:03.228175+0000","last_peered":"2024-03-16T08:15:03.228175+0000","last_clean":"2024-03-16T08:15:03.228175+0000","last_became_active":"2024-03-16T08:15:02.174417+0000","last_became_peered":"2024-03-16T08:15:02.174417+0000","last_unstale":"2024-03-16T08:15:03.228175+0000","last_undegraded":"2024-03-16T08:15:03.228175+0000","last_fullsized":"2024-03-16T08:15:03.228175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:00:39.444643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154614+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:15:03.154614+0000","last_peered":"2024-03-16T08:15:03.154614+0000","last_clean":"2024-03-16T08:15:03.154614+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:15:03.154614+0000","last_undegraded":"2024-03-16T08:15:03.154614+0000","last_fullsized":"2024-03-16T08:15:03.154614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226848+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:15:03.226848+0000","last_peered":"2024-03-16T08:15:03.226848+0000","last_clean":"2024-03-16T08:15:03.226848+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:15:03.226848+0000","last_undegraded":"2024-03-16T08:15:03.226848+0000","last_fullsized":"2024-03-16T08:15:03.226848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153596+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:15:03.153596+0000","last_peered":"2024-03-16T08:15:03.153596+0000","last_clean":"2024-03-16T08:15:03.153596+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:15:03.153596+0000","last_undegraded":"2024-03-16T08:15:03.153596+0000","last_fullsized":"2024-03-16T08:15:03.153596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.564790+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:15:05.564790+0000","last_peered":"2024-03-16T08:15:05.564790+0000","last_clean":"2024-03-16T08:15:05.564790+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:15:05.564790+0000","last_undegraded":"2024-03-16T08:15:05.564790+0000","last_fullsized":"2024-03-16T08:15:05.564790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227097+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:15:03.227097+0000","last_peered":"2024-03-16T08:15:03.227097+0000","last_clean":"2024-03-16T08:15:03.227097+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:15:03.227097+0000","last_undegraded":"2024-03-16T08:15:03.227097+0000","last_fullsized":"2024-03-16T08:15:03.227097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226754+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:15:03.226754+0000","last_peered":"2024-03-16T08:15:03.226754+0000","last_clean":"2024-03-16T08:15:03.226754+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:15:03.226754+0000","last_undegraded":"2024-03-16T08:15:03.226754+0000","last_fullsized":"2024-03-16T08:15:03.226754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227311+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:15:03.227311+0000","last_peered":"2024-03-16T08:15:03.227311+0000","last_clean":"2024-03-16T08:15:03.227311+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:15:03.227311+0000","last_undegraded":"2024-03-16T08:15:03.227311+0000","last_fullsized":"2024-03-16T08:15:03.227311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243917+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:15:03.243917+0000","last_peered":"2024-03-16T08:15:03.243917+0000","last_clean":"2024-03-16T08:15:03.243917+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:15:03.243917+0000","last_undegraded":"2024-03-16T08:15:03.243917+0000","last_fullsized":"2024-03-16T08:15:03.243917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227207+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:15:03.227207+0000","last_peered":"2024-03-16T08:15:03.227207+0000","last_clean":"2024-03-16T08:15:03.227207+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:15:03.227207+0000","last_undegraded":"2024-03-16T08:15:03.227207+0000","last_fullsized":"2024-03-16T08:15:03.227207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.575072+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:15:06.575072+0000","last_peered":"2024-03-16T08:15:06.575072+0000","last_clean":"2024-03-16T08:15:06.575072+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:15:06.575072+0000","last_undegraded":"2024-03-16T08:15:06.575072+0000","last_fullsized":"2024-03-16T08:15:06.575072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153809+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:15:03.153809+0000","last_peered":"2024-03-16T08:15:03.153809+0000","last_clean":"2024-03-16T08:15:03.153809+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:15:03.153809+0000","last_undegraded":"2024-03-16T08:15:03.153809+0000","last_fullsized":"2024-03-16T08:15:03.153809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.564020+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:15:05.564020+0000","last_peered":"2024-03-16T08:15:05.564020+0000","last_clean":"2024-03-16T08:15:05.564020+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:15:05.564020+0000","last_undegraded":"2024-03-16T08:15:05.564020+0000","last_fullsized":"2024-03-16T08:15:05.564020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226576+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:15:03.226576+0000","last_peered":"2024-03-16T08:15:03.226576+0000","last_clean":"2024-03-16T08:15:03.226576+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:15:03.226576+0000","last_undegraded":"2024-03-16T08:15:03.226576+0000","last_fullsized":"2024-03-16T08:15:03.226576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.565215+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:15:05.565215+0000","last_peered":"2024-03-16T08:15:05.565215+0000","last_clean":"2024-03-16T08:15:05.565215+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:15:05.565215+0000","last_undegraded":"2024-03-16T08:15:05.565215+0000","last_fullsized":"2024-03-16T08:15:05.565215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243527+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:15:03.243527+0000","last_peered":"2024-03-16T08:15:03.243527+0000","last_clean":"2024-03-16T08:15:03.243527+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:15:03.243527+0000","last_undegraded":"2024-03-16T08:15:03.243527+0000","last_fullsized":"2024-03-16T08:15:03.243527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566738+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:15:05.566738+0000","last_peered":"2024-03-16T08:15:05.566738+0000","last_clean":"2024-03-16T08:15:05.566738+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:15:05.566738+0000","last_undegraded":"2024-03-16T08:15:05.566738+0000","last_fullsized":"2024-03-16T08:15:05.566738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226987+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:15:03.226987+0000","last_peered":"2024-03-16T08:15:03.226987+0000","last_clean":"2024-03-16T08:15:03.226987+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:15:03.226987+0000","last_undegraded":"2024-03-16T08:15:03.226987+0000","last_fullsized":"2024-03-16T08:15:03.226987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226405+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:15:03.226405+0000","last_peered":"2024-03-16T08:15:03.226405+0000","last_clean":"2024-03-16T08:15:03.226405+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:15:03.226405+0000","last_undegraded":"2024-03-16T08:15:03.226405+0000","last_fullsized":"2024-03-16T08:15:03.226405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153658+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:15:03.153658+0000","last_peered":"2024-03-16T08:15:03.153658+0000","last_clean":"2024-03-16T08:15:03.153658+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:15:03.153658+0000","last_undegraded":"2024-03-16T08:15:03.153658+0000","last_fullsized":"2024-03-16T08:15:03.153658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566843+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:15:05.566843+0000","last_peered":"2024-03-16T08:15:05.566843+0000","last_clean":"2024-03-16T08:15:05.566843+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:15:05.566843+0000","last_undegraded":"2024-03-16T08:15:05.566843+0000","last_fullsized":"2024-03-16T08:15:05.566843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227011+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:15:03.227011+0000","last_peered":"2024-03-16T08:15:03.227011+0000","last_clean":"2024-03-16T08:15:03.227011+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:15:03.227011+0000","last_undegraded":"2024-03-16T08:15:03.227011+0000","last_fullsized":"2024-03-16T08:15:03.227011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.573424+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:15:06.573424+0000","last_peered":"2024-03-16T08:15:06.573424+0000","last_clean":"2024-03-16T08:15:06.573424+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:15:06.573424+0000","last_undegraded":"2024-03-16T08:15:06.573424+0000","last_fullsized":"2024-03-16T08:15:06.573424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227950+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:15:03.227950+0000","last_peered":"2024-03-16T08:15:03.227950+0000","last_clean":"2024-03-16T08:15:03.227950+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:15:03.227950+0000","last_undegraded":"2024-03-16T08:15:03.227950+0000","last_fullsized":"2024-03-16T08:15:03.227950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154387+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:15:03.154387+0000","last_peered":"2024-03-16T08:15:03.154387+0000","last_clean":"2024-03-16T08:15:03.154387+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:15:03.154387+0000","last_undegraded":"2024-03-16T08:15:03.154387+0000","last_fullsized":"2024-03-16T08:15:03.154387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227189+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:15:03.227189+0000","last_peered":"2024-03-16T08:15:03.227189+0000","last_clean":"2024-03-16T08:15:03.227189+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:15:03.227189+0000","last_undegraded":"2024-03-16T08:15:03.227189+0000","last_fullsized":"2024-03-16T08:15:03.227189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227359+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:15:03.227359+0000","last_peered":"2024-03-16T08:15:03.227359+0000","last_clean":"2024-03-16T08:15:03.227359+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:15:03.227359+0000","last_undegraded":"2024-03-16T08:15:03.227359+0000","last_fullsized":"2024-03-16T08:15:03.227359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228119+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:15:03.228119+0000","last_peered":"2024-03-16T08:15:03.228119+0000","last_clean":"2024-03-16T08:15:03.228119+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:15:03.228119+0000","last_undegraded":"2024-03-16T08:15:03.228119+0000","last_fullsized":"2024-03-16T08:15:03.228119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227207+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:15:03.227207+0000","last_peered":"2024-03-16T08:15:03.227207+0000","last_clean":"2024-03-16T08:15:03.227207+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:15:03.227207+0000","last_undegraded":"2024-03-16T08:15:03.227207+0000","last_fullsized":"2024-03-16T08:15:03.227207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226455+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:15:03.226455+0000","last_peered":"2024-03-16T08:15:03.226455+0000","last_clean":"2024-03-16T08:15:03.226455+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:15:03.226455+0000","last_undegraded":"2024-03-16T08:15:03.226455+0000","last_fullsized":"2024-03-16T08:15:03.226455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226497+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:15:03.226497+0000","last_peered":"2024-03-16T08:15:03.226497+0000","last_clean":"2024-03-16T08:15:03.226497+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:15:03.226497+0000","last_undegraded":"2024-03-16T08:15:03.226497+0000","last_fullsized":"2024-03-16T08:15:03.226497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226911+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:15:03.226911+0000","last_peered":"2024-03-16T08:15:03.226911+0000","last_clean":"2024-03-16T08:15:03.226911+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:15:03.226911+0000","last_undegraded":"2024-03-16T08:15:03.226911+0000","last_fullsized":"2024-03-16T08:15:03.226911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227305+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:15:03.227305+0000","last_peered":"2024-03-16T08:15:03.227305+0000","last_clean":"2024-03-16T08:15:03.227305+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:15:03.227305+0000","last_undegraded":"2024-03-16T08:15:03.227305+0000","last_fullsized":"2024-03-16T08:15:03.227305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227239+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:15:03.227239+0000","last_peered":"2024-03-16T08:15:03.227239+0000","last_clean":"2024-03-16T08:15:03.227239+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:15:03.227239+0000","last_undegraded":"2024-03-16T08:15:03.227239+0000","last_fullsized":"2024-03-16T08:15:03.227239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227197+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:15:03.227197+0000","last_peered":"2024-03-16T08:15:03.227197+0000","last_clean":"2024-03-16T08:15:03.227197+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:15:03.227197+0000","last_undegraded":"2024-03-16T08:15:03.227197+0000","last_fullsized":"2024-03-16T08:15:03.227197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153789+0000","last_change":"2024-03-16T08:15:02.174276+0000","last_active":"2024-03-16T08:15:03.153789+0000","last_peered":"2024-03-16T08:15:03.153789+0000","last_clean":"2024-03-16T08:15:03.153789+0000","last_became_active":"2024-03-16T08:15:02.169937+0000","last_became_peered":"2024-03-16T08:15:02.169937+0000","last_unstale":"2024-03-16T08:15:03.153789+0000","last_undegraded":"2024-03-16T08:15:03.153789+0000","last_fullsized":"2024-03-16T08:15:03.153789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:31.496674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154322+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:15:03.154322+0000","last_peered":"2024-03-16T08:15:03.154322+0000","last_clean":"2024-03-16T08:15:03.154322+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:15:03.154322+0000","last_undegraded":"2024-03-16T08:15:03.154322+0000","last_fullsized":"2024-03-16T08:15:03.154322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153579+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:15:03.153579+0000","last_peered":"2024-03-16T08:15:03.153579+0000","last_clean":"2024-03-16T08:15:03.153579+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:15:03.153579+0000","last_undegraded":"2024-03-16T08:15:03.153579+0000","last_fullsized":"2024-03-16T08:15:03.153579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227811+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:15:03.227811+0000","last_peered":"2024-03-16T08:15:03.227811+0000","last_clean":"2024-03-16T08:15:03.227811+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:15:03.227811+0000","last_undegraded":"2024-03-16T08:15:03.227811+0000","last_fullsized":"2024-03-16T08:15:03.227811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227300+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:15:03.227300+0000","last_peered":"2024-03-16T08:15:03.227300+0000","last_clean":"2024-03-16T08:15:03.227300+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:15:03.227300+0000","last_undegraded":"2024-03-16T08:15:03.227300+0000","last_fullsized":"2024-03-16T08:15:03.227300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226929+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:15:03.226929+0000","last_peered":"2024-03-16T08:15:03.226929+0000","last_clean":"2024-03-16T08:15:03.226929+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:15:03.226929+0000","last_undegraded":"2024-03-16T08:15:03.226929+0000","last_fullsized":"2024-03-16T08:15:03.226929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4398,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":27444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":449784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":16812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:15:10.399 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3230313848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d641023a0 msgr2=0x7f2d64104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3230313848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d641023a0 0x7f2d64104790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009860 tx=0x7f2d5802bb70 comp rx=0 tx=0).stop 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3230313848 shutdown_connections 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3230313848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64107600 0x7f2d641099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3230313848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d64104cd0 0x7f2d641070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3230313848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d641023a0 0x7f2d64104790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3230313848 >> 172.21.15.49:0/3230313848 conn(0x7f2d640fbd10 msgr2=0x7f2d640fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3230313848 shutdown_connections 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3230313848 wait complete. 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 Processor -- start 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.544+0000 7f2d6c0b2640 1 -- start start 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64104cd0 0x7f2d64198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d64107600 0x7f2d6419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410d480 con 0x7f2d641023a0 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410d300 con 0x7f2d64104cd0 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410d600 con 0x7f2d64107600 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41592/0 (socket says 172.21.15.49:41592) 2024-03-16T08:15:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 -- 172.21.15.49:0/3955350756 learned_addr learned my addr 172.21.15.49:0/3955350756 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6a64f640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d64107600 0x7f2d6419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6964d640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64104cd0 0x7f2d64198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 -- 172.21.15.49:0/3955350756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d64107600 msgr2=0x7f2d6419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d64107600 0x7f2d6419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 -- 172.21.15.49:0/3955350756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64104cd0 msgr2=0x7f2d64198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64104cd0 0x7f2d64198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 -- 172.21.15.49:0/3955350756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6419f750 con 0x7f2d641023a0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d69e4e640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005940 tx=0x7f2d58032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58041020 con 0x7f2d641023a0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6419f9e0 con 0x7f2d641023a0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5802c9f0 con 0x7f2d641023a0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.545+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58039040 con 0x7f2d641023a0 2024-03-16T08:15:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.546+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6419ff20 con 0x7f2d641023a0 2024-03-16T08:15:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.546+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d640fd800 con 0x7f2d641023a0 2024-03-16T08:15:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.547+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2d58044020 con 0x7f2d641023a0 2024-03-16T08:15:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.548+0000 7f2d56ffd640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 0x7f2d500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.548+0000 7f2d6964d640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 0x7f2d500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.548+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2d580f02f0 con 0x7f2d641023a0 2024-03-16T08:15:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.548+0000 7f2d6964d640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 0x7f2d500a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d64199040 tx=0x7f2d60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.552+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2d5804b020 con 0x7f2d641023a0 2024-03-16T08:15:10.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.733+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d64061c20 con 0x7f2d500a2890 2024-03-16T08:15:10.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.740+0000 7f2d56ffd640 1 -- 172.21.15.49:0/3955350756 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258857 (secure 0 0 0) 0x7f2d64061c20 con 0x7f2d500a2890 2024-03-16T08:15:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:10.744 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 msgr2=0x7f2d500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 0x7f2d500a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d64199040 tx=0x7f2d60009260 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 msgr2=0x7f2d64197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005940 tx=0x7f2d58032470 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 shutdown_connections 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d64107600 0x7f2d6419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d64104cd0 0x7f2d64198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2d500a2890 0x7f2d500a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 --2- 172.21.15.49:0/3955350756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d641023a0 0x7f2d64197b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.743+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 >> 172.21.15.49:0/3955350756 conn(0x7f2d640fbd10 msgr2=0x7f2d64102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:10.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.744+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 shutdown_connections 2024-03-16T08:15:10.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:10.744+0000 7f2d6c0b2640 1 -- 172.21.15.49:0/3955350756 wait complete. 2024-03-16T08:15:10.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:10 smithi049 ceph-mon[31942]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 12 op/s 2024-03-16T08:15:10.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:10 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/638328027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:10 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/264337341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:10 smithi049 ceph-mon[31942]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:10.755 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-16T08:15:09.580770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594766,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":281,"num_write_kb":2965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356140,"kb_used_data":11112,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585684,"statfs":{"total":1151940427776,"available":1151575740416,"internally_reserved":0,"allocated":11378688,"data_stored":7088062,"data_compressed":108399,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4398,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001741"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227166+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:15:03.227166+0000","last_peered":"2024-03-16T08:15:03.227166+0000","last_clean":"2024-03-16T08:15:03.227166+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:15:03.227166+0000","last_undegraded":"2024-03-16T08:15:03.227166+0000","last_fullsized":"2024-03-16T08:15:03.227166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226759+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:15:03.226759+0000","last_peered":"2024-03-16T08:15:03.226759+0000","last_clean":"2024-03-16T08:15:03.226759+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:15:03.226759+0000","last_undegraded":"2024-03-16T08:15:03.226759+0000","last_fullsized":"2024-03-16T08:15:03.226759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227962+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:15:03.227962+0000","last_peered":"2024-03-16T08:15:03.227962+0000","last_clean":"2024-03-16T08:15:03.227962+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:15:03.227962+0000","last_undegraded":"2024-03-16T08:15:03.227962+0000","last_fullsized":"2024-03-16T08:15:03.227962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154943+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:15:03.154943+0000","last_peered":"2024-03-16T08:15:03.154943+0000","last_clean":"2024-03-16T08:15:03.154943+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:15:03.154943+0000","last_undegraded":"2024-03-16T08:15:03.154943+0000","last_fullsized":"2024-03-16T08:15:03.154943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243590+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:15:03.243590+0000","last_peered":"2024-03-16T08:15:03.243590+0000","last_clean":"2024-03-16T08:15:03.243590+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:15:03.243590+0000","last_undegraded":"2024-03-16T08:15:03.243590+0000","last_fullsized":"2024-03-16T08:15:03.243590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227148+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:15:03.227148+0000","last_peered":"2024-03-16T08:15:03.227148+0000","last_clean":"2024-03-16T08:15:03.227148+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:15:03.227148+0000","last_undegraded":"2024-03-16T08:15:03.227148+0000","last_fullsized":"2024-03-16T08:15:03.227148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227784+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:15:03.227784+0000","last_peered":"2024-03-16T08:15:03.227784+0000","last_clean":"2024-03-16T08:15:03.227784+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:15:03.227784+0000","last_undegraded":"2024-03-16T08:15:03.227784+0000","last_fullsized":"2024-03-16T08:15:03.227784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153230+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:15:03.153230+0000","last_peered":"2024-03-16T08:15:03.153230+0000","last_clean":"2024-03-16T08:15:03.153230+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:15:03.153230+0000","last_undegraded":"2024-03-16T08:15:03.153230+0000","last_fullsized":"2024-03-16T08:15:03.153230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227764+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:15:03.227764+0000","last_peered":"2024-03-16T08:15:03.227764+0000","last_clean":"2024-03-16T08:15:03.227764+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:15:03.227764+0000","last_undegraded":"2024-03-16T08:15:03.227764+0000","last_fullsized":"2024-03-16T08:15:03.227764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227212+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:15:03.227212+0000","last_peered":"2024-03-16T08:15:03.227212+0000","last_clean":"2024-03-16T08:15:03.227212+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:15:03.227212+0000","last_undegraded":"2024-03-16T08:15:03.227212+0000","last_fullsized":"2024-03-16T08:15:03.227212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228196+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:15:03.228196+0000","last_peered":"2024-03-16T08:15:03.228196+0000","last_clean":"2024-03-16T08:15:03.228196+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:15:03.228196+0000","last_undegraded":"2024-03-16T08:15:03.228196+0000","last_fullsized":"2024-03-16T08:15:03.228196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154911+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:15:03.154911+0000","last_peered":"2024-03-16T08:15:03.154911+0000","last_clean":"2024-03-16T08:15:03.154911+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:15:03.154911+0000","last_undegraded":"2024-03-16T08:15:03.154911+0000","last_fullsized":"2024-03-16T08:15:03.154911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226996+0000","last_change":"2024-03-16T08:15:02.168429+0000","last_active":"2024-03-16T08:15:03.226996+0000","last_peered":"2024-03-16T08:15:03.226996+0000","last_clean":"2024-03-16T08:15:03.226996+0000","last_became_active":"2024-03-16T08:15:02.167835+0000","last_became_peered":"2024-03-16T08:15:02.167835+0000","last_unstale":"2024-03-16T08:15:03.226996+0000","last_undegraded":"2024-03-16T08:15:03.226996+0000","last_fullsized":"2024-03-16T08:15:03.226996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:00:22.118862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228200+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:15:03.228200+0000","last_peered":"2024-03-16T08:15:03.228200+0000","last_clean":"2024-03-16T08:15:03.228200+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:15:03.228200+0000","last_undegraded":"2024-03-16T08:15:03.228200+0000","last_fullsized":"2024-03-16T08:15:03.228200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226497+0000","last_change":"2024-03-16T08:15:02.175165+0000","last_active":"2024-03-16T08:15:03.226497+0000","last_peered":"2024-03-16T08:15:03.226497+0000","last_clean":"2024-03-16T08:15:03.226497+0000","last_became_active":"2024-03-16T08:15:02.173186+0000","last_became_peered":"2024-03-16T08:15:02.173186+0000","last_unstale":"2024-03-16T08:15:03.226497+0000","last_undegraded":"2024-03-16T08:15:03.226497+0000","last_fullsized":"2024-03-16T08:15:03.226497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:28.302644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227637+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:15:03.227637+0000","last_peered":"2024-03-16T08:15:03.227637+0000","last_clean":"2024-03-16T08:15:03.227637+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:15:03.227637+0000","last_undegraded":"2024-03-16T08:15:03.227637+0000","last_fullsized":"2024-03-16T08:15:03.227637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227177+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:15:03.227177+0000","last_peered":"2024-03-16T08:15:03.227177+0000","last_clean":"2024-03-16T08:15:03.227177+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:15:03.227177+0000","last_undegraded":"2024-03-16T08:15:03.227177+0000","last_fullsized":"2024-03-16T08:15:03.227177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226389+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:15:03.226389+0000","last_peered":"2024-03-16T08:15:03.226389+0000","last_clean":"2024-03-16T08:15:03.226389+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:15:03.226389+0000","last_undegraded":"2024-03-16T08:15:03.226389+0000","last_fullsized":"2024-03-16T08:15:03.226389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227336+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:15:03.227336+0000","last_peered":"2024-03-16T08:15:03.227336+0000","last_clean":"2024-03-16T08:15:03.227336+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:15:03.227336+0000","last_undegraded":"2024-03-16T08:15:03.227336+0000","last_fullsized":"2024-03-16T08:15:03.227336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.575498+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:15:06.575498+0000","last_peered":"2024-03-16T08:15:06.575498+0000","last_clean":"2024-03-16T08:15:06.575498+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:15:06.575498+0000","last_undegraded":"2024-03-16T08:15:06.575498+0000","last_fullsized":"2024-03-16T08:15:06.575498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153790+0000","last_change":"2024-03-16T08:15:02.169282+0000","last_active":"2024-03-16T08:15:03.153790+0000","last_peered":"2024-03-16T08:15:03.153790+0000","last_clean":"2024-03-16T08:15:03.153790+0000","last_became_active":"2024-03-16T08:15:02.168163+0000","last_became_peered":"2024-03-16T08:15:02.168163+0000","last_unstale":"2024-03-16T08:15:03.153790+0000","last_undegraded":"2024-03-16T08:15:03.153790+0000","last_fullsized":"2024-03-16T08:15:03.153790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:00:05.780516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154014+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:15:03.154014+0000","last_peered":"2024-03-16T08:15:03.154014+0000","last_clean":"2024-03-16T08:15:03.154014+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:15:03.154014+0000","last_undegraded":"2024-03-16T08:15:03.154014+0000","last_fullsized":"2024-03-16T08:15:03.154014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227452+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:15:03.227452+0000","last_peered":"2024-03-16T08:15:03.227452+0000","last_clean":"2024-03-16T08:15:03.227452+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:15:03.227452+0000","last_undegraded":"2024-03-16T08:15:03.227452+0000","last_fullsized":"2024-03-16T08:15:03.227452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243968+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:15:03.243968+0000","last_peered":"2024-03-16T08:15:03.243968+0000","last_clean":"2024-03-16T08:15:03.243968+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:15:03.243968+0000","last_undegraded":"2024-03-16T08:15:03.243968+0000","last_fullsized":"2024-03-16T08:15:03.243968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227920+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:15:03.227920+0000","last_peered":"2024-03-16T08:15:03.227920+0000","last_clean":"2024-03-16T08:15:03.227920+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:15:03.227920+0000","last_undegraded":"2024-03-16T08:15:03.227920+0000","last_fullsized":"2024-03-16T08:15:03.227920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243690+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:15:03.243690+0000","last_peered":"2024-03-16T08:15:03.243690+0000","last_clean":"2024-03-16T08:15:03.243690+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:15:03.243690+0000","last_undegraded":"2024-03-16T08:15:03.243690+0000","last_fullsized":"2024-03-16T08:15:03.243690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154563+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:15:03.154563+0000","last_peered":"2024-03-16T08:15:03.154563+0000","last_clean":"2024-03-16T08:15:03.154563+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:15:03.154563+0000","last_undegraded":"2024-03-16T08:15:03.154563+0000","last_fullsized":"2024-03-16T08:15:03.154563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.567546+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:15:05.567546+0000","last_peered":"2024-03-16T08:15:05.567546+0000","last_clean":"2024-03-16T08:15:05.567546+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:15:05.567546+0000","last_undegraded":"2024-03-16T08:15:05.567546+0000","last_fullsized":"2024-03-16T08:15:05.567546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243906+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:15:03.243906+0000","last_peered":"2024-03-16T08:15:03.243906+0000","last_clean":"2024-03-16T08:15:03.243906+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:15:03.243906+0000","last_undegraded":"2024-03-16T08:15:03.243906+0000","last_fullsized":"2024-03-16T08:15:03.243906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.574344+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:15:06.574344+0000","last_peered":"2024-03-16T08:15:06.574344+0000","last_clean":"2024-03-16T08:15:06.574344+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:15:06.574344+0000","last_undegraded":"2024-03-16T08:15:06.574344+0000","last_fullsized":"2024-03-16T08:15:06.574344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154514+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:15:03.154514+0000","last_peered":"2024-03-16T08:15:03.154514+0000","last_clean":"2024-03-16T08:15:03.154514+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:15:03.154514+0000","last_undegraded":"2024-03-16T08:15:03.154514+0000","last_fullsized":"2024-03-16T08:15:03.154514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226935+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:15:03.226935+0000","last_peered":"2024-03-16T08:15:03.226935+0000","last_clean":"2024-03-16T08:15:03.226935+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:15:03.226935+0000","last_undegraded":"2024-03-16T08:15:03.226935+0000","last_fullsized":"2024-03-16T08:15:03.226935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153319+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:15:03.153319+0000","last_peered":"2024-03-16T08:15:03.153319+0000","last_clean":"2024-03-16T08:15:03.153319+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:15:03.153319+0000","last_undegraded":"2024-03-16T08:15:03.153319+0000","last_fullsized":"2024-03-16T08:15:03.153319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153942+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:15:03.153942+0000","last_peered":"2024-03-16T08:15:03.153942+0000","last_clean":"2024-03-16T08:15:03.153942+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:15:03.153942+0000","last_undegraded":"2024-03-16T08:15:03.153942+0000","last_fullsized":"2024-03-16T08:15:03.153942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.225995+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:15:03.225995+0000","last_peered":"2024-03-16T08:15:03.225995+0000","last_clean":"2024-03-16T08:15:03.225995+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:15:03.225995+0000","last_undegraded":"2024-03-16T08:15:03.225995+0000","last_fullsized":"2024-03-16T08:15:03.225995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566528+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:15:05.566528+0000","last_peered":"2024-03-16T08:15:05.566528+0000","last_clean":"2024-03-16T08:15:05.566528+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:15:05.566528+0000","last_undegraded":"2024-03-16T08:15:05.566528+0000","last_fullsized":"2024-03-16T08:15:05.566528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227528+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:15:03.227528+0000","last_peered":"2024-03-16T08:15:03.227528+0000","last_clean":"2024-03-16T08:15:03.227528+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:15:03.227528+0000","last_undegraded":"2024-03-16T08:15:03.227528+0000","last_fullsized":"2024-03-16T08:15:03.227528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227303+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:15:03.227303+0000","last_peered":"2024-03-16T08:15:03.227303+0000","last_clean":"2024-03-16T08:15:03.227303+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:15:03.227303+0000","last_undegraded":"2024-03-16T08:15:03.227303+0000","last_fullsized":"2024-03-16T08:15:03.227303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227620+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:15:03.227620+0000","last_peered":"2024-03-16T08:15:03.227620+0000","last_clean":"2024-03-16T08:15:03.227620+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:15:03.227620+0000","last_undegraded":"2024-03-16T08:15:03.227620+0000","last_fullsized":"2024-03-16T08:15:03.227620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154614+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:15:03.154614+0000","last_peered":"2024-03-16T08:15:03.154614+0000","last_clean":"2024-03-16T08:15:03.154614+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:15:03.154614+0000","last_undegraded":"2024-03-16T08:15:03.154614+0000","last_fullsized":"2024-03-16T08:15:03.154614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228077+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:15:03.228077+0000","last_peered":"2024-03-16T08:15:03.228077+0000","last_clean":"2024-03-16T08:15:03.228077+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:15:03.228077+0000","last_undegraded":"2024-03-16T08:15:03.228077+0000","last_fullsized":"2024-03-16T08:15:03.228077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.565481+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:15:05.565481+0000","last_peered":"2024-03-16T08:15:05.565481+0000","last_clean":"2024-03-16T08:15:05.565481+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:15:05.565481+0000","last_undegraded":"2024-03-16T08:15:05.565481+0000","last_fullsized":"2024-03-16T08:15:05.565481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243773+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:15:03.243773+0000","last_peered":"2024-03-16T08:15:03.243773+0000","last_clean":"2024-03-16T08:15:03.243773+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:15:03.243773+0000","last_undegraded":"2024-03-16T08:15:03.243773+0000","last_fullsized":"2024-03-16T08:15:03.243773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.571589+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:15:06.571589+0000","last_peered":"2024-03-16T08:15:06.571589+0000","last_clean":"2024-03-16T08:15:06.571589+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:15:06.571589+0000","last_undegraded":"2024-03-16T08:15:06.571589+0000","last_fullsized":"2024-03-16T08:15:06.571589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227203+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:15:03.227203+0000","last_peered":"2024-03-16T08:15:03.227203+0000","last_clean":"2024-03-16T08:15:03.227203+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:15:03.227203+0000","last_undegraded":"2024-03-16T08:15:03.227203+0000","last_fullsized":"2024-03-16T08:15:03.227203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566759+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:15:05.566759+0000","last_peered":"2024-03-16T08:15:05.566759+0000","last_clean":"2024-03-16T08:15:05.566759+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:15:05.566759+0000","last_undegraded":"2024-03-16T08:15:05.566759+0000","last_fullsized":"2024-03-16T08:15:05.566759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227567+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:15:03.227567+0000","last_peered":"2024-03-16T08:15:03.227567+0000","last_clean":"2024-03-16T08:15:03.227567+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:15:03.227567+0000","last_undegraded":"2024-03-16T08:15:03.227567+0000","last_fullsized":"2024-03-16T08:15:03.227567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154417+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:15:03.154417+0000","last_peered":"2024-03-16T08:15:03.154417+0000","last_clean":"2024-03-16T08:15:03.154417+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:15:03.154417+0000","last_undegraded":"2024-03-16T08:15:03.154417+0000","last_fullsized":"2024-03-16T08:15:03.154417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227040+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:15:03.227040+0000","last_peered":"2024-03-16T08:15:03.227040+0000","last_clean":"2024-03-16T08:15:03.227040+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:15:03.227040+0000","last_undegraded":"2024-03-16T08:15:03.227040+0000","last_fullsized":"2024-03-16T08:15:03.227040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566316+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:15:05.566316+0000","last_peered":"2024-03-16T08:15:05.566316+0000","last_clean":"2024-03-16T08:15:05.566316+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:15:05.566316+0000","last_undegraded":"2024-03-16T08:15:05.566316+0000","last_fullsized":"2024-03-16T08:15:05.566316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154239+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:15:03.154239+0000","last_peered":"2024-03-16T08:15:03.154239+0000","last_clean":"2024-03-16T08:15:03.154239+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:15:03.154239+0000","last_undegraded":"2024-03-16T08:15:03.154239+0000","last_fullsized":"2024-03-16T08:15:03.154239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243685+0000","last_change":"2024-03-16T08:15:01.171487+0000","last_active":"2024-03-16T08:15:03.243685+0000","last_peered":"2024-03-16T08:15:03.243685+0000","last_clean":"2024-03-16T08:15:03.243685+0000","last_became_active":"2024-03-16T08:15:01.171164+0000","last_became_peered":"2024-03-16T08:15:01.171164+0000","last_unstale":"2024-03-16T08:15:03.243685+0000","last_undegraded":"2024-03-16T08:15:03.243685+0000","last_fullsized":"2024-03-16T08:15:03.243685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:49:52.984972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227049+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:15:03.227049+0000","last_peered":"2024-03-16T08:15:03.227049+0000","last_clean":"2024-03-16T08:15:03.227049+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:15:03.227049+0000","last_undegraded":"2024-03-16T08:15:03.227049+0000","last_fullsized":"2024-03-16T08:15:03.227049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226025+0000","last_change":"2024-03-16T08:15:01.168022+0000","last_active":"2024-03-16T08:15:03.226025+0000","last_peered":"2024-03-16T08:15:03.226025+0000","last_clean":"2024-03-16T08:15:03.226025+0000","last_became_active":"2024-03-16T08:15:01.167316+0000","last_became_peered":"2024-03-16T08:15:01.167316+0000","last_unstale":"2024-03-16T08:15:03.226025+0000","last_undegraded":"2024-03-16T08:15:03.226025+0000","last_fullsized":"2024-03-16T08:15:03.226025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:08:56.392344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227292+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:15:03.227292+0000","last_peered":"2024-03-16T08:15:03.227292+0000","last_clean":"2024-03-16T08:15:03.227292+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:15:03.227292+0000","last_undegraded":"2024-03-16T08:15:03.227292+0000","last_fullsized":"2024-03-16T08:15:03.227292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226814+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:15:03.226814+0000","last_peered":"2024-03-16T08:15:03.226814+0000","last_clean":"2024-03-16T08:15:03.226814+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:15:03.226814+0000","last_undegraded":"2024-03-16T08:15:03.226814+0000","last_fullsized":"2024-03-16T08:15:03.226814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228175+0000","last_change":"2024-03-16T08:15:02.174717+0000","last_active":"2024-03-16T08:15:03.228175+0000","last_peered":"2024-03-16T08:15:03.228175+0000","last_clean":"2024-03-16T08:15:03.228175+0000","last_became_active":"2024-03-16T08:15:02.174417+0000","last_became_peered":"2024-03-16T08:15:02.174417+0000","last_unstale":"2024-03-16T08:15:03.228175+0000","last_undegraded":"2024-03-16T08:15:03.228175+0000","last_fullsized":"2024-03-16T08:15:03.228175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:00:39.444643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154614+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:15:03.154614+0000","last_peered":"2024-03-16T08:15:03.154614+0000","last_clean":"2024-03-16T08:15:03.154614+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:15:03.154614+0000","last_undegraded":"2024-03-16T08:15:03.154614+0000","last_fullsized":"2024-03-16T08:15:03.154614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226848+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:15:03.226848+0000","last_peered":"2024-03-16T08:15:03.226848+0000","last_clean":"2024-03-16T08:15:03.226848+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:15:03.226848+0000","last_undegraded":"2024-03-16T08:15:03.226848+0000","last_fullsized":"2024-03-16T08:15:03.226848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153596+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:15:03.153596+0000","last_peered":"2024-03-16T08:15:03.153596+0000","last_clean":"2024-03-16T08:15:03.153596+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:15:03.153596+0000","last_undegraded":"2024-03-16T08:15:03.153596+0000","last_fullsized":"2024-03-16T08:15:03.153596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.564790+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:15:05.564790+0000","last_peered":"2024-03-16T08:15:05.564790+0000","last_clean":"2024-03-16T08:15:05.564790+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:15:05.564790+0000","last_undegraded":"2024-03-16T08:15:05.564790+0000","last_fullsized":"2024-03-16T08:15:05.564790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227097+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:15:03.227097+0000","last_peered":"2024-03-16T08:15:03.227097+0000","last_clean":"2024-03-16T08:15:03.227097+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:15:03.227097+0000","last_undegraded":"2024-03-16T08:15:03.227097+0000","last_fullsized":"2024-03-16T08:15:03.227097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226754+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:15:03.226754+0000","last_peered":"2024-03-16T08:15:03.226754+0000","last_clean":"2024-03-16T08:15:03.226754+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:15:03.226754+0000","last_undegraded":"2024-03-16T08:15:03.226754+0000","last_fullsized":"2024-03-16T08:15:03.226754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227311+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:15:03.227311+0000","last_peered":"2024-03-16T08:15:03.227311+0000","last_clean":"2024-03-16T08:15:03.227311+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:15:03.227311+0000","last_undegraded":"2024-03-16T08:15:03.227311+0000","last_fullsized":"2024-03-16T08:15:03.227311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243917+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:15:03.243917+0000","last_peered":"2024-03-16T08:15:03.243917+0000","last_clean":"2024-03-16T08:15:03.243917+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:15:03.243917+0000","last_undegraded":"2024-03-16T08:15:03.243917+0000","last_fullsized":"2024-03-16T08:15:03.243917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227207+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:15:03.227207+0000","last_peered":"2024-03-16T08:15:03.227207+0000","last_clean":"2024-03-16T08:15:03.227207+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:15:03.227207+0000","last_undegraded":"2024-03-16T08:15:03.227207+0000","last_fullsized":"2024-03-16T08:15:03.227207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.575072+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:15:06.575072+0000","last_peered":"2024-03-16T08:15:06.575072+0000","last_clean":"2024-03-16T08:15:06.575072+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:15:06.575072+0000","last_undegraded":"2024-03-16T08:15:06.575072+0000","last_fullsized":"2024-03-16T08:15:06.575072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153809+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:15:03.153809+0000","last_peered":"2024-03-16T08:15:03.153809+0000","last_clean":"2024-03-16T08:15:03.153809+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:15:03.153809+0000","last_undegraded":"2024-03-16T08:15:03.153809+0000","last_fullsized":"2024-03-16T08:15:03.153809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.564020+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:15:05.564020+0000","last_peered":"2024-03-16T08:15:05.564020+0000","last_clean":"2024-03-16T08:15:05.564020+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:15:05.564020+0000","last_undegraded":"2024-03-16T08:15:05.564020+0000","last_fullsized":"2024-03-16T08:15:05.564020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226576+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:15:03.226576+0000","last_peered":"2024-03-16T08:15:03.226576+0000","last_clean":"2024-03-16T08:15:03.226576+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:15:03.226576+0000","last_undegraded":"2024-03-16T08:15:03.226576+0000","last_fullsized":"2024-03-16T08:15:03.226576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.565215+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:15:05.565215+0000","last_peered":"2024-03-16T08:15:05.565215+0000","last_clean":"2024-03-16T08:15:05.565215+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:15:05.565215+0000","last_undegraded":"2024-03-16T08:15:05.565215+0000","last_fullsized":"2024-03-16T08:15:05.565215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.243527+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:15:03.243527+0000","last_peered":"2024-03-16T08:15:03.243527+0000","last_clean":"2024-03-16T08:15:03.243527+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:15:03.243527+0000","last_undegraded":"2024-03-16T08:15:03.243527+0000","last_fullsized":"2024-03-16T08:15:03.243527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566738+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:15:05.566738+0000","last_peered":"2024-03-16T08:15:05.566738+0000","last_clean":"2024-03-16T08:15:05.566738+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:15:05.566738+0000","last_undegraded":"2024-03-16T08:15:05.566738+0000","last_fullsized":"2024-03-16T08:15:05.566738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226987+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:15:03.226987+0000","last_peered":"2024-03-16T08:15:03.226987+0000","last_clean":"2024-03-16T08:15:03.226987+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:15:03.226987+0000","last_undegraded":"2024-03-16T08:15:03.226987+0000","last_fullsized":"2024-03-16T08:15:03.226987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226405+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:15:03.226405+0000","last_peered":"2024-03-16T08:15:03.226405+0000","last_clean":"2024-03-16T08:15:03.226405+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:15:03.226405+0000","last_undegraded":"2024-03-16T08:15:03.226405+0000","last_fullsized":"2024-03-16T08:15:03.226405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153658+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:15:03.153658+0000","last_peered":"2024-03-16T08:15:03.153658+0000","last_clean":"2024-03-16T08:15:03.153658+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:15:03.153658+0000","last_undegraded":"2024-03-16T08:15:03.153658+0000","last_fullsized":"2024-03-16T08:15:03.153658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:05.566843+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:15:05.566843+0000","last_peered":"2024-03-16T08:15:05.566843+0000","last_clean":"2024-03-16T08:15:05.566843+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:15:05.566843+0000","last_undegraded":"2024-03-16T08:15:05.566843+0000","last_fullsized":"2024-03-16T08:15:05.566843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227011+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:15:03.227011+0000","last_peered":"2024-03-16T08:15:03.227011+0000","last_clean":"2024-03-16T08:15:03.227011+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:15:03.227011+0000","last_undegraded":"2024-03-16T08:15:03.227011+0000","last_fullsized":"2024-03-16T08:15:03.227011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:06.573424+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:15:06.573424+0000","last_peered":"2024-03-16T08:15:06.573424+0000","last_clean":"2024-03-16T08:15:06.573424+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:15:06.573424+0000","last_undegraded":"2024-03-16T08:15:06.573424+0000","last_fullsized":"2024-03-16T08:15:06.573424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227950+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:15:03.227950+0000","last_peered":"2024-03-16T08:15:03.227950+0000","last_clean":"2024-03-16T08:15:03.227950+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:15:03.227950+0000","last_undegraded":"2024-03-16T08:15:03.227950+0000","last_fullsized":"2024-03-16T08:15:03.227950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154387+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:15:03.154387+0000","last_peered":"2024-03-16T08:15:03.154387+0000","last_clean":"2024-03-16T08:15:03.154387+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:15:03.154387+0000","last_undegraded":"2024-03-16T08:15:03.154387+0000","last_fullsized":"2024-03-16T08:15:03.154387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227189+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:15:03.227189+0000","last_peered":"2024-03-16T08:15:03.227189+0000","last_clean":"2024-03-16T08:15:03.227189+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:15:03.227189+0000","last_undegraded":"2024-03-16T08:15:03.227189+0000","last_fullsized":"2024-03-16T08:15:03.227189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227359+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:15:03.227359+0000","last_peered":"2024-03-16T08:15:03.227359+0000","last_clean":"2024-03-16T08:15:03.227359+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:15:03.227359+0000","last_undegraded":"2024-03-16T08:15:03.227359+0000","last_fullsized":"2024-03-16T08:15:03.227359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.228119+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:15:03.228119+0000","last_peered":"2024-03-16T08:15:03.228119+0000","last_clean":"2024-03-16T08:15:03.228119+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:15:03.228119+0000","last_undegraded":"2024-03-16T08:15:03.228119+0000","last_fullsized":"2024-03-16T08:15:03.228119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227207+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:15:03.227207+0000","last_peered":"2024-03-16T08:15:03.227207+0000","last_clean":"2024-03-16T08:15:03.227207+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:15:03.227207+0000","last_undegraded":"2024-03-16T08:15:03.227207+0000","last_fullsized":"2024-03-16T08:15:03.227207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226455+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:15:03.226455+0000","last_peered":"2024-03-16T08:15:03.226455+0000","last_clean":"2024-03-16T08:15:03.226455+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:15:03.226455+0000","last_undegraded":"2024-03-16T08:15:03.226455+0000","last_fullsized":"2024-03-16T08:15:03.226455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226497+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:15:03.226497+0000","last_peered":"2024-03-16T08:15:03.226497+0000","last_clean":"2024-03-16T08:15:03.226497+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:15:03.226497+0000","last_undegraded":"2024-03-16T08:15:03.226497+0000","last_fullsized":"2024-03-16T08:15:03.226497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226911+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:15:03.226911+0000","last_peered":"2024-03-16T08:15:03.226911+0000","last_clean":"2024-03-16T08:15:03.226911+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:15:03.226911+0000","last_undegraded":"2024-03-16T08:15:03.226911+0000","last_fullsized":"2024-03-16T08:15:03.226911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227305+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:15:03.227305+0000","last_peered":"2024-03-16T08:15:03.227305+0000","last_clean":"2024-03-16T08:15:03.227305+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:15:03.227305+0000","last_undegraded":"2024-03-16T08:15:03.227305+0000","last_fullsized":"2024-03-16T08:15:03.227305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227239+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:15:03.227239+0000","last_peered":"2024-03-16T08:15:03.227239+0000","last_clean":"2024-03-16T08:15:03.227239+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:15:03.227239+0000","last_undegraded":"2024-03-16T08:15:03.227239+0000","last_fullsized":"2024-03-16T08:15:03.227239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227197+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:15:03.227197+0000","last_peered":"2024-03-16T08:15:03.227197+0000","last_clean":"2024-03-16T08:15:03.227197+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:15:03.227197+0000","last_undegraded":"2024-03-16T08:15:03.227197+0000","last_fullsized":"2024-03-16T08:15:03.227197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153789+0000","last_change":"2024-03-16T08:15:02.174276+0000","last_active":"2024-03-16T08:15:03.153789+0000","last_peered":"2024-03-16T08:15:03.153789+0000","last_clean":"2024-03-16T08:15:03.153789+0000","last_became_active":"2024-03-16T08:15:02.169937+0000","last_became_peered":"2024-03-16T08:15:02.169937+0000","last_unstale":"2024-03-16T08:15:03.153789+0000","last_undegraded":"2024-03-16T08:15:03.153789+0000","last_fullsized":"2024-03-16T08:15:03.153789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:31.496674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.154322+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:15:03.154322+0000","last_peered":"2024-03-16T08:15:03.154322+0000","last_clean":"2024-03-16T08:15:03.154322+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:15:03.154322+0000","last_undegraded":"2024-03-16T08:15:03.154322+0000","last_fullsized":"2024-03-16T08:15:03.154322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.153579+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:15:03.153579+0000","last_peered":"2024-03-16T08:15:03.153579+0000","last_clean":"2024-03-16T08:15:03.153579+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:15:03.153579+0000","last_undegraded":"2024-03-16T08:15:03.153579+0000","last_fullsized":"2024-03-16T08:15:03.153579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227811+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:15:03.227811+0000","last_peered":"2024-03-16T08:15:03.227811+0000","last_clean":"2024-03-16T08:15:03.227811+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:15:03.227811+0000","last_undegraded":"2024-03-16T08:15:03.227811+0000","last_fullsized":"2024-03-16T08:15:03.227811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.227300+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:15:03.227300+0000","last_peered":"2024-03-16T08:15:03.227300+0000","last_clean":"2024-03-16T08:15:03.227300+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:15:03.227300+0000","last_undegraded":"2024-03-16T08:15:03.227300+0000","last_fullsized":"2024-03-16T08:15:03.227300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-16T08:15:03.226929+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:15:03.226929+0000","last_peered":"2024-03-16T08:15:03.226929+0000","last_clean":"2024-03-16T08:15:03.226929+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:15:03.226929+0000","last_undegraded":"2024-03-16T08:15:03.226929+0000","last_fullsized":"2024-03-16T08:15:03.226929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4398,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":27444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":449784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":16812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:15:10.760 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T08:15:10.760 INFO:tasks.cephadm:Setting up client nodes... 2024-03-16T08:15:10.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T08:15:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:10 smithi129 ceph-mon[36561]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 12 op/s 2024-03-16T08:15:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:10 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/638328027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:10 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/264337341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:10 smithi129 ceph-mon[36561]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:10.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:10 smithi154 ceph-mon[36471]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 12 op/s 2024-03-16T08:15:10.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:10 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/638328027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:10 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/264337341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:10.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:10 smithi154 ceph-mon[36471]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:11.046 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:11.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:11 smithi049 ceph-mon[31942]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:11 smithi129 ceph-mon[36561]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:11.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:11 smithi154 ceph-mon[36471]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 -- 172.21.15.49:0/3552249631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 msgr2=0x7f2cfc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3552249631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1028b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0099c0 tx=0x7f2cec009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 -- 172.21.15.49:0/3552249631 shutdown_connections 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3552249631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 0x7f2cfc1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3552249631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3552249631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1028b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 -- 172.21.15.49:0/3552249631 >> 172.21.15.49:0/3552249631 conn(0x7f2cfc0fc7f0 msgr2=0x7f2cfc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 -- 172.21.15.49:0/3552249631 shutdown_connections 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.717+0000 7f2d032c0700 1 -- 172.21.15.49:0/3552249631 wait complete. 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 Processor -- start 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 -- start start 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 0x7f2cfc19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10ce60 con 0x7f2cfc102450 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10cce0 con 0x7f2cfc101250 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d032c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10cfe0 con 0x7f2cfc102df0 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51608/0 (socket says 172.21.15.49:51608) 2024-03-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 -- 172.21.15.49:0/3252920076 learned_addr learned my addr 172.21.15.49:0/3252920076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0105c700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0185d700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 0x7f2cfc19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 -- 172.21.15.49:0/3252920076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 msgr2=0x7f2cfc19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 0x7f2cfc19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 -- 172.21.15.49:0/3252920076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 msgr2=0x7f2cfc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0085b700 1 -- 172.21.15.49:0/3252920076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a00f0 con 0x7f2cfc102450 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.718+0000 7f2d0105c700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2d0085b700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005910 tx=0x7f2cec00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec009e60 con 0x7f2cfc102450 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cec00f460 con 0x7f2cfc102450 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec017650 con 0x7f2cfc102450 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a0380 con 0x7f2cfc102450 2024-03-16T08:15:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.719+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a0830 con 0x7f2cfc102450 2024-03-16T08:15:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.720+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc0507e0 con 0x7f2cfc102450 2024-03-16T08:15:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.721+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2cec00f910 con 0x7f2cfc102450 2024-03-16T08:15:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.721+0000 7f2cf27fc700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 0x7f2ce80a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.722+0000 7f2d0105c700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 0x7f2ce80a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.722+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2cec0cf7e0 con 0x7f2cfc102450 2024-03-16T08:15:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.722+0000 7f2d0105c700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 0x7f2ce80a4f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc1022b0 tx=0x7f2cf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:12.726+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2cec013070 con 0x7f2cfc102450 2024-03-16T08:15:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:12 smithi049 ceph-mon[31942]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-16T08:15:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:12 smithi129 ceph-mon[36561]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-16T08:15:12.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:12 smithi154 ceph-mon[36471]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-16T08:15:13.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.015+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2cfc1a0db0 con 0x7f2cfc102450 2024-03-16T08:15:13.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.017+0000 7f2cf27fc700 1 -- 172.21.15.49:0/3252920076 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f2cec09c960 con 0x7f2cfc102450 2024-03-16T08:15:13.017 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-03-16T08:15:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQARVfVllcb6ABAAHZvm9ocZricT9CV4kVeYsw== 2024-03-16T08:15:13.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 msgr2=0x7f2ce80a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:13.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 0x7f2ce80a4f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc1022b0 tx=0x7f2cf8009260 comp rx=0 tx=0).stop 2024-03-16T08:15:13.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 msgr2=0x7f2cfc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:13.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005910 tx=0x7f2cec00bee0 comp rx=0 tx=0).stop 2024-03-16T08:15:13.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 shutdown_connections 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ce80a2a50 0x7f2ce80a4f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102df0 0x7f2cfc19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cfc101250 0x7f2cfc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 --2- 172.21.15.49:0/3252920076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cfc102450 0x7f2cfc1989d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 >> 172.21.15.49:0/3252920076 conn(0x7f2cfc0fc7f0 msgr2=0x7f2cfc06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 shutdown_connections 2024-03-16T08:15:13.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:13.020+0000 7f2d032c0700 1 -- 172.21.15.49:0/3252920076 wait complete. 2024-03-16T08:15:13.622 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:15:13.622 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-16T08:15:13.622 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T08:15:13.671 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T08:15:13.671 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-16T08:15:13.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-16T08:15:13.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:13 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:15:13.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:13 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:15:13.876 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:13 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:15:13.876 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:13 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:15:13.962 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:14.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:13 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:15:14.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:13 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3252920076' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:15:15.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:15 smithi049 ceph-mon[31942]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-16T08:15:15.145 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:15 smithi154 ceph-mon[36471]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-16T08:15:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:15 smithi129 ceph-mon[36561]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/1395250907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0101b70 msgr2=0x7f7ed010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/1395250907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0101b70 0x7f7ed010e050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009990 tx=0x7f7ec0009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/1395250907 shutdown_connections 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/1395250907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0101b70 0x7f7ed010e050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/1395250907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0107210 0x7f7ed01075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/1395250907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed01011d0 0x7f7ed0101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/1395250907 >> 172.21.15.49:0/1395250907 conn(0x7f7ed0078100 msgr2=0x7f7ed0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/1395250907 shutdown_connections 2024-03-16T08:15:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.614+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/1395250907 wait complete. 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.615+0000 7f7ed4e5b700 1 Processor -- start 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.615+0000 7f7ed4e5b700 1 -- start start 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01011d0 0x7f7ed0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed0101b70 0x7f7ed0199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111490 con 0x7f7ed0107210 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111310 con 0x7f7ed01011d0 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ed4e5b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111610 con 0x7f7ed0101b70 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ece59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01011d0 0x7f7ed0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51630/0 (socket says 172.21.15.49:51630) 2024-03-16T08:15:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ecdd9b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed0101b70 0x7f7ed0199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 -- 172.21.15.49:0/4134416900 learned_addr learned my addr 172.21.15.49:0/4134416900 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 -- 172.21.15.49:0/4134416900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed0101b70 msgr2=0x7f7ed0199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed0101b70 0x7f7ed0199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 -- 172.21.15.49:0/4134416900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01011d0 msgr2=0x7f7ed0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01011d0 0x7f7ed0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 -- 172.21.15.49:0/4134416900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01a4e10 con 0x7f7ed0107210 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7eced9d700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00038b0 tx=0x7f7ec0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.616+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0027070 con 0x7f7ed0107210 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.617+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a5040 con 0x7f7ed0107210 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.617+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0003e40 con 0x7f7ed0107210 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.617+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0022cc0 con 0x7f7ed0107210 2024-03-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.617+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a5490 con 0x7f7ed0107210 2024-03-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.617+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0068f90 con 0x7f7ed0107210 2024-03-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.619+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7ec002a070 con 0x7f7ed0107210 2024-03-16T08:15:15.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.620+0000 7f7ec77fe700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 0x7f7ebc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:15.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.620+0000 7f7ece59c700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 0x7f7ebc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:15.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.620+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7ec00cffa0 con 0x7f7ed0107210 2024-03-16T08:15:15.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.620+0000 7f7ece59c700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 0x7f7ebc0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009550 tx=0x7f7ec80091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:15.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.623+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7ec0031020 con 0x7f7ed0107210 2024-03-16T08:15:15.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.887+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7ed002ddd0 con 0x7f7ed0107210 2024-03-16T08:15:15.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.892+0000 7f7ec77fe700 1 -- 172.21.15.49:0/4134416900 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262660 (secure 0 0 0) 0x7f7ec0013070 con 0x7f7ed0107210 2024-03-16T08:15:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:15.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 msgr2=0x7f7ebc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:15.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 0x7f7ebc0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009550 tx=0x7f7ec80091f0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 msgr2=0x7f7ed01a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00038b0 tx=0x7f7ec0005e80 comp rx=0 tx=0).stop 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 shutdown_connections 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7ebc0a2920 0x7f7ebc0a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed0107210 0x7f7ed01a48d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01011d0 0x7f7ed0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 --2- 172.21.15.49:0/4134416900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ed0101b70 0x7f7ed0199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 >> 172.21.15.49:0/4134416900 conn(0x7f7ed0078100 msgr2=0x7f7ed00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 shutdown_connections 2024-03-16T08:15:15.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:15.896+0000 7f7ed4e5b700 1 -- 172.21.15.49:0/4134416900 wait complete. 2024-03-16T08:15:16.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:16 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4134416900' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T08:15:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:16 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4134416900' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T08:15:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:16 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4134416900' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T08:15:16.570 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":2256169095},{"type":"v1","addr":"172.21.15.49:6801","nonce":2256169095}]},"active_addr":"172.21.15.49:6801/2256169095","active_change":"2024-03-16T08:08:03.498598+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":236450725}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1561370229}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3786569188}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3762364594}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":4122153811}]}]} 2024-03-16T08:15:16.576 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-16T08:15:16.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-16T08:15:16.576 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:15:16.834 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:17.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:17 smithi049 ceph-mon[31942]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T08:15:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:17 smithi129 ceph-mon[36561]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T08:15:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:17 smithi154 ceph-mon[36471]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 -- 172.21.15.49:0/1948235662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd974101cc0 msgr2=0x7fd974105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 --2- 172.21.15.49:0/1948235662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd974101cc0 0x7fd974105d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9680099c0 tx=0x7fd968009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 -- 172.21.15.49:0/1948235662 shutdown_connections 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 --2- 172.21.15.49:0/1948235662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9741063f0 0x7fd97410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 --2- 172.21.15.49:0/1948235662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101310 0x7fd9741016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 --2- 172.21.15.49:0/1948235662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd974101cc0 0x7fd974105d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 -- 172.21.15.49:0/1948235662 >> 172.21.15.49:0/1948235662 conn(0x7fd9740fcba0 msgr2=0x7fd9740fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 -- 172.21.15.49:0/1948235662 shutdown_connections 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 -- 172.21.15.49:0/1948235662 wait complete. 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.615+0000 7fd97359e700 1 Processor -- start 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 -- start start 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 0x7fd97419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 0x7fd9741987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741113b0 con 0x7fd974101cc0 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111230 con 0x7fd974101310 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111530 con 0x7fd9741063f0 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd971d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd971d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51650/0 (socket says 172.21.15.49:51650) 2024-03-16T08:15:18.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd971d9b700 1 -- 172.21.15.49:0/3406238082 learned_addr learned my addr 172.21.15.49:0/3406238082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 0x7fd97419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd972d9d700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 0x7fd9741987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 -- 172.21.15.49:0/3406238082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 msgr2=0x7fd9741987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 0x7fd9741987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 -- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 msgr2=0x7fd97419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 -- 172.21.15.49:0/3406238082 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd974198f90 con 0x7fd974101310 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd971d9b700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd972d9d700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 0x7fd9741987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.616+0000 7fd97259c700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 0x7fd97419e630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.617+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400cb70 con 0x7fd974101310 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.617+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd974199280 con 0x7fd974101310 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.617+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96400ee90 con 0x7fd974101310 2024-03-16T08:15:18.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.617+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964003dc0 con 0x7fd974101310 2024-03-16T08:15:18.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.617+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9741a5490 con 0x7fd974101310 2024-03-16T08:15:18.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.618+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9741027f0 con 0x7fd974101310 2024-03-16T08:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.619+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd964003f60 con 0x7fd974101310 2024-03-16T08:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.620+0000 7fd9637fe700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 0x7fd95c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.620+0000 7fd971d9b700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 0x7fd95c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.620+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd9640ced30 con 0x7fd974101310 2024-03-16T08:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.620+0000 7fd971d9b700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 0x7fd95c0a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ff1c0 tx=0x7fd968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:18.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.628+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd96409bdb0 con 0x7fd974101310 2024-03-16T08:15:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.832+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd974199ce0 con 0x7fd974101310 2024-03-16T08:15:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.833+0000 7fd9637fe700 1 -- 172.21.15.49:0/3406238082 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22184 (secure 0 0 0) 0x7fd96409b6b0 con 0x7fd974101310 2024-03-16T08:15:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:18.834 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:15:03.149983+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.835+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 msgr2=0x7fd95c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.835+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 0x7fd95c0a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ff1c0 tx=0x7fd968005fb0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.835+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 msgr2=0x7fd97419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.835+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 0x7fd97419e630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400ed00 comp rx=0 tx=0).stop 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 shutdown_connections 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd95c0a2920 0x7fd95c0a4de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9741063f0 0x7fd9741987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd974101310 0x7fd97419e630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 --2- 172.21.15.49:0/3406238082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd974101cc0 0x7fd97419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 >> 172.21.15.49:0/3406238082 conn(0x7fd9740fcba0 msgr2=0x7fd97406abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 shutdown_connections 2024-03-16T08:15:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:18.836+0000 7fd97359e700 1 -- 172.21.15.49:0/3406238082 wait complete. 2024-03-16T08:15:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:19 smithi049 ceph-mon[31942]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-03-16T08:15:19.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:19 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3406238082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:19 smithi129 ceph-mon[36561]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-03-16T08:15:19.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:19 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3406238082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:19 smithi154 ceph-mon[36471]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-03-16T08:15:19.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:19 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3406238082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:19.479 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-16T08:15:19.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:15:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:21.177 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:21 smithi049 ceph-mon[31942]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-03-16T08:15:21.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:21 smithi129 ceph-mon[36561]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- 172.21.15.49:0/4102229331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa601064e0 msgr2=0x7ffa6010e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4102229331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa601064e0 0x7ffa6010e0b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800b0d0 tx=0x7ffa5800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- 172.21.15.49:0/4102229331 shutdown_connections 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4102229331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa601064e0 0x7ffa6010e0b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4102229331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60101490 0x7ffa60101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4102229331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101e40 0x7ffa60105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- 172.21.15.49:0/4102229331 >> 172.21.15.49:0/4102229331 conn(0x7ffa60077220 msgr2=0x7ffa60077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:21.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- 172.21.15.49:0/4102229331 shutdown_connections 2024-03-16T08:15:21.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- 172.21.15.49:0/4102229331 wait complete. 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 Processor -- start 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.372+0000 7ffa66b63700 1 -- start start 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 0x7ffa601a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 0x7ffa601a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010fb60 con 0x7ffa601064e0 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010f9e0 con 0x7ffa60101490 2024-03-16T08:15:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa66b63700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010fce0 con 0x7ffa60101e40 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa648ff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 0x7ffa601a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51670/0 (socket says 172.21.15.49:51670) 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 -- 172.21.15.49:0/4115722317 learned_addr learned my addr 172.21.15.49:0/4115722317 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa5ffff700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 0x7ffa601a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 -- 172.21.15.49:0/4115722317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 msgr2=0x7ffa601a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 0x7ffa601a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 -- 172.21.15.49:0/4115722317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 msgr2=0x7ffa601a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 0x7ffa601a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 -- 172.21.15.49:0/4115722317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6019b190 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa5ffff700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 0x7ffa601a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa648ff700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 0x7ffa601a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.373+0000 7ffa65100700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006070 tx=0x7ffa58007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.374+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58003c00 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.374+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6019b420 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.374+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa58022070 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.374+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801b6c0 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.374+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a7740 con 0x7ffa601064e0 2024-03-16T08:15:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.375+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa60068f90 con 0x7ffa601064e0 2024-03-16T08:15:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.376+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ffa5801d470 con 0x7ffa601064e0 2024-03-16T08:15:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.376+0000 7ffa5dffb700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 0x7ffa4c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.377+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffa5800f020 con 0x7ffa601064e0 2024-03-16T08:15:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.377+0000 7ffa648ff700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 0x7ffa4c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.377+0000 7ffa648ff700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 0x7ffa4c0a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffa6019bb70 tx=0x7ffa48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:21.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.381+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffa5809c8c0 con 0x7ffa601064e0 2024-03-16T08:15:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:21 smithi154 ceph-mon[36471]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-03-16T08:15:21.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.589+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa601a7910 con 0x7ffa601064e0 2024-03-16T08:15:21.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.590+0000 7ffa5dffb700 1 -- 172.21.15.49:0/4115722317 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22184 (secure 0 0 0) 0x7ffa5809c1c0 con 0x7ffa601064e0 2024-03-16T08:15:21.592 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:15:21.592 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:15:03.149983+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:15:21.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 msgr2=0x7ffa4c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 0x7ffa4c0a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffa6019bb70 tx=0x7ffa48009260 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 msgr2=0x7ffa6019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006070 tx=0x7ffa58007a20 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 shutdown_connections 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffa4c0a2920 0x7ffa4c0a4de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa601064e0 0x7ffa6019aa30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa60101490 0x7ffa601a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 --2- 172.21.15.49:0/4115722317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa60101e40 0x7ffa601a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 >> 172.21.15.49:0/4115722317 conn(0x7ffa60077220 msgr2=0x7ffa600fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 shutdown_connections 2024-03-16T08:15:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:21.592+0000 7ffa66b63700 1 -- 172.21.15.49:0/4115722317 wait complete. 2024-03-16T08:15:22.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:22 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4115722317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:22.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-16T08:15:22.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-16T08:15:22.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-16T08:15:22.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-16T08:15:22.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-16T08:15:22.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-16T08:15:22.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-16T08:15:22.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-16T08:15:22.298 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-16T08:15:22.298 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-16T08:15:22.299 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-16T08:15:22.299 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-16T08:15:22.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:22 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4115722317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:22 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4115722317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:15:23.048 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:23.052 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:23 smithi049 ceph-mon[31942]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-03-16T08:15:23.123 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:23.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:23 smithi129 ceph-mon[36561]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-03-16T08:15:23.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:23 smithi154 ceph-mon[36471]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-03-16T08:15:24.066 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.319 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:25 smithi049 ceph-mon[31942]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-03-16T08:15:25.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:25 smithi129 ceph-mon[36561]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-03-16T08:15:25.425 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:25 smithi154 ceph-mon[36471]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-03-16T08:15:25.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 -- 172.21.15.49:0/4110407851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406b420 msgr2=0x7fd164106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 --2- 172.21.15.49:0/4110407851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406b420 0x7fd164106f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd158009990 tx=0x7fd158009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 -- 172.21.15.49:0/4110407851 shutdown_connections 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 --2- 172.21.15.49:0/4110407851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1641075e0 0x7fd16410f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 --2- 172.21.15.49:0/4110407851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd16406aa70 0x7fd16406ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 --2- 172.21.15.49:0/4110407851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406b420 0x7fd164106f00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 -- 172.21.15.49:0/4110407851 >> 172.21.15.49:0/4110407851 conn(0x7fd164077220 msgr2=0x7fd164077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 -- 172.21.15.49:0/4110407851 shutdown_connections 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.656+0000 7fd16a995700 1 -- 172.21.15.49:0/4110407851 wait complete. 2024-03-16T08:15:25.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 Processor -- start 2024-03-16T08:15:25.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 -- start start 2024-03-16T08:15:25.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 0x7fd1641012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:25.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16406b420 0x7fd164101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:25.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1641075e0 0x7fd164101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd164112580 con 0x7fd16406aa70 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd164112400 con 0x7fd1641075e0 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd16a995700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd164112700 con 0x7fd16406b420 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd163fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 0x7fd1641012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd168f32700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1641075e0 0x7fd164101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd163fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 0x7fd1641012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51390/0 (socket says 172.21.15.49:51390) 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd163fff700 1 -- 172.21.15.49:0/732363144 learned_addr learned my addr 172.21.15.49:0/732363144 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.657+0000 7fd1637fe700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16406b420 0x7fd164101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd1637fe700 1 -- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1641075e0 msgr2=0x7fd164101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd1637fe700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1641075e0 0x7fd164101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd1637fe700 1 -- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 msgr2=0x7fd1641012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd1637fe700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 0x7fd1641012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd1637fe700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1641a6f50 con 0x7fd16406b420 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.658+0000 7fd163fff700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd16406aa70 0x7fd1641012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd1637fe700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16406b420 0x7fd164101810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd1580098e0 tx=0x7fd15800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158027070 con 0x7fd16406b420 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158003d70 con 0x7fd16406b420 2024-03-16T08:15:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15800f460 con 0x7fd16406b420 2024-03-16T08:15:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1641a71e0 con 0x7fd16406b420 2024-03-16T08:15:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.659+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a7640 con 0x7fd16406b420 2024-03-16T08:15:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.661+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd164062a60 con 0x7fd16406b420 2024-03-16T08:15:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.661+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd15802a070 con 0x7fd16406b420 2024-03-16T08:15:25.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.662+0000 7fd1617fa700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd14c0a29a0 0x7fd14c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:25.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd1580cf9f0 con 0x7fd16406b420 2024-03-16T08:15:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd1617fa700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] conn(0x7fd14c0ac180 0x7fd14c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd15802be50 con 0x7fd14c0ac180 2024-03-16T08:15:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd1580d42a0 con 0x7fd16406b420 2024-03-16T08:15:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd163fff700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd14c0a29a0 0x7fd14c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:25.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.663+0000 7fd168f32700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] conn(0x7fd14c0ac180 0x7fd14c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:25.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.664+0000 7fd163fff700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd14c0a29a0 0x7fd14c0a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd154009d70 tx=0x7fd154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:25.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.664+0000 7fd168f32700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] conn(0x7fd14c0ac180 0x7fd14c0ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:25.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.665+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== osd.4 v2:172.21.15.129:6808/615229289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd15802be50 con 0x7fd14c0ac180 2024-03-16T08:15:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.683+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 --> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd164062e60 con 0x7fd14c0ac180 2024-03-16T08:15:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.683+0000 7fd1617fa700 1 -- 172.21.15.49:0/732363144 <== osd.4 v2:172.21.15.129:6808/615229289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd164062e60 con 0x7fd14c0ac180 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] conn(0x7fd14c0ac180 msgr2=0x7fd14c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] conn(0x7fd14c0ac180 0x7fd14c0ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd14c0a29a0 msgr2=0x7fd14c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd14c0a29a0 0x7fd14c0a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd154009d70 tx=0x7fd154009260 comp rx=0 tx=0).stop 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16406b420 msgr2=0x7fd164101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 --2- 172.21.15.49:0/732363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16406b420 0x7fd164101810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd1580098e0 tx=0x7fd15800bab0 comp rx=0 tx=0).stop 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd168f32700 1 -- 172.21.15.49:0/732363144 reap_dead start 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 shutdown_connections 2024-03-16T08:15:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 >> 172.21.15.49:0/732363144 conn(0x7fd164077220 msgr2=0x7fd1640ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:25.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 shutdown_connections 2024-03-16T08:15:25.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:25.684+0000 7fd16a995700 1 -- 172.21.15.49:0/732363144 wait complete. 2024-03-16T08:15:27.297 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:27 smithi049 ceph-mon[31942]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:15:27.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:27 smithi129 ceph-mon[36561]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:15:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:27 smithi154 ceph-mon[36471]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:15:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:146028888117 2024-03-16T08:15:28.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-16T08:15:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:29.341 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:29 smithi049 ceph-mon[31942]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T08:15:29.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:29 smithi129 ceph-mon[36561]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T08:15:29.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:29 smithi154 ceph-mon[36471]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 -- 172.21.15.49:0/4099066649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 msgr2=0x7ff7f406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/4099066649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f406a3e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00099c0 tx=0x7ff7e0009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 -- 172.21.15.49:0/4099066649 shutdown_connections 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/4099066649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406a920 0x7ff7f410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/4099066649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f4108320 0x7ff7f4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/4099066649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f406a3e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 -- 172.21.15.49:0/4099066649 >> 172.21.15.49:0/4099066649 conn(0x7ff7f4078100 msgr2=0x7ff7f4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 -- 172.21.15.49:0/4099066649 shutdown_connections 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.256+0000 7ff7faf0a700 1 -- 172.21.15.49:0/4099066649 wait complete. 2024-03-16T08:15:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 Processor -- start 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 -- start start 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f406a920 0x7ff7f419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f4108320 0x7ff7f4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41125a0 con 0x7ff7f4069f80 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4112420 con 0x7ff7f406a920 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7faf0a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4112720 con 0x7ff7f4108320 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7f8ca6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7f8ca6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51406/0 (socket says 172.21.15.49:51406) 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7f8ca6700 1 -- 172.21.15.49:0/912721296 learned_addr learned my addr 172.21.15.49:0/912721296 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7f3fff700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f406a920 0x7ff7f419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.257+0000 7ff7f94a7700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f4108320 0x7ff7f4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 -- 172.21.15.49:0/912721296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f4108320 msgr2=0x7ff7f4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f4108320 0x7ff7f4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 -- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f406a920 msgr2=0x7ff7f419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f406a920 0x7ff7f419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4198fc0 con 0x7ff7f4069f80 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f8ca6700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f419e6a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9a0 tx=0x7ff7e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800ca10 con 0x7ff7f4069f80 2024-03-16T08:15:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41992b0 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e800cbb0 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8018680 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.258+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41a5570 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.259+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7f4062a60 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.260+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff7e8010430 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f1ffb700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7e40a2920 0x7ff7e40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f3fff700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7e40a2920 0x7ff7e40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff7e80cf3f0 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f1ffb700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] conn(0x7ff7e40ac100 0x7ff7e40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7e801bec0 con 0x7ff7e40ac100 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.261+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff7e80cf8d0 con 0x7ff7f4069f80 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.262+0000 7ff7f94a7700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] conn(0x7ff7e40ac100 0x7ff7e40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.262+0000 7ff7f3fff700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7e40a2920 0x7ff7e40a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4078770 tx=0x7ff7e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.262+0000 7ff7f94a7700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] conn(0x7ff7e40ac100 0x7ff7e40ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.263+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== osd.1 v2:172.21.15.129:6800/646273559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7e801bec0 con 0x7ff7e40ac100 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.279+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 --> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7f4062ee0 con 0x7ff7e40ac100 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.279+0000 7ff7f1ffb700 1 -- 172.21.15.49:0/912721296 <== osd.1 v2:172.21.15.129:6800/646273559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff7f4062ee0 con 0x7ff7e40ac100 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.279+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] conn(0x7ff7e40ac100 msgr2=0x7ff7e40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.279+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] conn(0x7ff7e40ac100 0x7ff7e40ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7e40a2920 msgr2=0x7ff7e40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7e40a2920 0x7ff7e40a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4078770 tx=0x7ff7e0005fb0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 msgr2=0x7ff7f419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 --2- 172.21.15.49:0/912721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7f4069f80 0x7ff7f419e6a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9a0 tx=0x7ff7e800ecb0 comp rx=0 tx=0).stop 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7f94a7700 1 -- 172.21.15.49:0/912721296 reap_dead start 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 shutdown_connections 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 >> 172.21.15.49:0/912721296 conn(0x7ff7f4078100 msgr2=0x7ff7f40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 shutdown_connections 2024-03-16T08:15:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:30.280+0000 7ff7faf0a700 1 -- 172.21.15.49:0/912721296 wait complete. 2024-03-16T08:15:31.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:31 smithi049 ceph-mon[31942]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:31.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:31 smithi129 ceph-mon[36561]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:31.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:31 smithi154 ceph-mon[36471]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:33.348 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:33 smithi049 ceph-mon[31942]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:33.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:33 smithi129 ceph-mon[36561]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:33.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:33 smithi154 ceph-mon[36471]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:35.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:35 smithi049 ceph-mon[31942]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:35.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:35 smithi129 ceph-mon[36561]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:35.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:35 smithi154 ceph-mon[36471]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.203+0000 7f72b28f6700 1 -- 172.21.15.49:0/2656298313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 msgr2=0x7f72ac102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 --2- 172.21.15.49:0/2656298313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac102990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f729c009990 tx=0x7f729c009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 -- 172.21.15.49:0/2656298313 shutdown_connections 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 --2- 172.21.15.49:0/2656298313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72ac107660 0x7f72ac10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 --2- 172.21.15.49:0/2656298313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac102990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 --2- 172.21.15.49:0/2656298313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ac102f60 0x7f72ac106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 -- 172.21.15.49:0/2656298313 >> 172.21.15.49:0/2656298313 conn(0x7f72ac0793a0 msgr2=0x7f72ac0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 -- 172.21.15.49:0/2656298313 shutdown_connections 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 -- 172.21.15.49:0/2656298313 wait complete. 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 Processor -- start 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.204+0000 7f72b28f6700 1 -- start start 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72ac102f60 0x7f72ac19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ac107660 0x7f72ac198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1125a0 con 0x7f72ac1025b0 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac112420 con 0x7f72ac102f60 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b28f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac112720 con 0x7f72ac107660 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34820/0 (socket says 172.21.15.49:34820) 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 -- 172.21.15.49:0/706156362 learned_addr learned my addr 172.21.15.49:0/706156362 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72ab7fe700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72ac102f60 0x7f72ac19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72b0e93700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ac107660 0x7f72ac198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 -- 172.21.15.49:0/706156362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ac107660 msgr2=0x7f72ac198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ac107660 0x7f72ac198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 -- 172.21.15.49:0/706156362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72ac102f60 msgr2=0x7f72ac19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72ac102f60 0x7f72ac19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac199000 con 0x7f72ac1025b0 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72abfff700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac19e650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f729c000c00 tx=0x7f729c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.205+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c027070 con 0x7f72ac1025b0 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.206+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac199290 con 0x7f72ac1025b0 2024-03-16T08:15:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.206+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c003d70 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.206+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c00f460 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.206+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a54b0 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.206+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72ac062a60 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f729c02a070 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72a97fa700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f72940a2920 0x7f72940a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72ab7fe700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f72940a2920 0x7f72940a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f729c0cfe20 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72a97fa700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] conn(0x7f72940ac100 0x7f72940ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.208+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72940aebd0 con 0x7f72940ac100 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.209+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f729c0d0300 con 0x7f72ac1025b0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.209+0000 7f72b0e93700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] conn(0x7f72940ac100 0x7f72940ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.209+0000 7f72ab7fe700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f72940a2920 0x7f72940a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72ac101e30 tx=0x7f72a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.209+0000 7f72b0e93700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] conn(0x7f72940ac100 0x7f72940ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:36.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.210+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== osd.3 v2:172.21.15.49:6810/1033582524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72940aebd0 con 0x7f72940ac100 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.226+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 --> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72ac062ee0 con 0x7f72940ac100 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.226+0000 7f72a97fa700 1 -- 172.21.15.49:0/706156362 <== osd.3 v2:172.21.15.49:6810/1033582524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f72ac062ee0 con 0x7f72940ac100 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] conn(0x7f72940ac100 msgr2=0x7f72940ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] conn(0x7f72940ac100 0x7f72940ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f72940a2920 msgr2=0x7f72940a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f72940a2920 0x7f72940a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72ac101e30 tx=0x7f72a0009260 comp rx=0 tx=0).stop 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 msgr2=0x7f72ac19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 --2- 172.21.15.49:0/706156362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ac1025b0 0x7f72ac19e650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f729c000c00 tx=0x7f729c00bab0 comp rx=0 tx=0).stop 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b0e93700 1 -- 172.21.15.49:0/706156362 reap_dead start 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 shutdown_connections 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 >> 172.21.15.49:0/706156362 conn(0x7f72ac0793a0 msgr2=0x7f72ac1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 shutdown_connections 2024-03-16T08:15:36.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:36.227+0000 7f72b28f6700 1 -- 172.21.15.49:0/706156362 wait complete. 2024-03-16T08:15:37.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:37 smithi049 ceph-mon[31942]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:37.234 INFO:teuthology.orchestra.run.smithi049.stdout:55834574917 2024-03-16T08:15:37.235 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-16T08:15:37.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:37 smithi129 ceph-mon[36561]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:37.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:37 smithi154 ceph-mon[36471]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:15:39.264 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:39 smithi049 ceph-mon[31942]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-16T08:15:39.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:39 smithi129 ceph-mon[36561]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-16T08:15:39.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:39 smithi154 ceph-mon[36471]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-16T08:15:39.952 INFO:teuthology.orchestra.run.smithi049.stdout:111669149754 2024-03-16T08:15:39.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-16T08:15:40.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 -- 172.21.15.49:0/2568157905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc102ec0 msgr2=0x7f73fc106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 --2- 172.21.15.49:0/2568157905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc102ec0 0x7f73fc106f10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0099c0 tx=0x7f73ec009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 -- 172.21.15.49:0/2568157905 shutdown_connections 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 --2- 172.21.15.49:0/2568157905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc1075f0 0x7f73fc10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 --2- 172.21.15.49:0/2568157905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73fc102510 0x7f73fc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 --2- 172.21.15.49:0/2568157905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc102ec0 0x7f73fc106f10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 -- 172.21.15.49:0/2568157905 >> 172.21.15.49:0/2568157905 conn(0x7f73fc0fdd80 msgr2=0x7f73fc1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 -- 172.21.15.49:0/2568157905 shutdown_connections 2024-03-16T08:15:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.134+0000 7f74047ee700 1 -- 172.21.15.49:0/2568157905 wait complete. 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 Processor -- start 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 -- start start 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 0x7f73fc0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73fc102ec0 0x7f73fc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc1075f0 0x7f73fc075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc1125b0 con 0x7f73fc1075f0 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc112430 con 0x7f73fc102510 2024-03-16T08:15:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f74047ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc112730 con 0x7f73fc102ec0 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f740258a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 0x7f73fc0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f7402d8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc1075f0 0x7f73fc075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f740258a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 0x7f73fc0745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:46146/0 (socket says 172.21.15.49:46146) 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f740258a700 1 -- 172.21.15.49:0/2777910909 learned_addr learned my addr 172.21.15.49:0/2777910909 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f7401d89700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73fc102ec0 0x7f73fc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.135+0000 7f7402d8b700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc1075f0 0x7f73fc075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34838/0 (socket says 172.21.15.49:34838) 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 -- 172.21.15.49:0/2777910909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73fc102ec0 msgr2=0x7f73fc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73fc102ec0 0x7f73fc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 -- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc1075f0 msgr2=0x7f73fc075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73fc1075f0 0x7f73fc075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc1a6f60 con 0x7f73fc102510 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f740258a700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 0x7f73fc0745d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9a0 tx=0x7f73f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800ca10 con 0x7f73fc102510 2024-03-16T08:15:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc1a7250 con 0x7f73fc102510 2024-03-16T08:15:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f800cbb0 con 0x7f73fc102510 2024-03-16T08:15:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8018680 con 0x7f73fc102510 2024-03-16T08:15:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.136+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc1a7760 con 0x7f73fc102510 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.137+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73fc0629e0 con 0x7f73fc102510 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.138+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f73f8010430 con 0x7f73fc102510 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f73f37fe700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73e80a2920 0x7f73e80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f73f80ced00 con 0x7f73fc102510 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f7401d89700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73e80a2920 0x7f73e80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f73f37fe700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] conn(0x7f73e80ac100 0x7f73e80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73f8012ec0 con 0x7f73e80ac100 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f73f80d13d0 con 0x7f73fc102510 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.139+0000 7f7402d8b700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] conn(0x7f73e80ac100 0x7f73e80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.140+0000 7f7401d89700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73e80a2920 0x7f73e80a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.140+0000 7f7402d8b700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] conn(0x7f73e80ac100 0x7f73e80ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.141+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== osd.10 v2:172.21.15.129:6824/2725967668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73f8012ec0 con 0x7f73e80ac100 2024-03-16T08:15:40.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.157+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 --> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73fc062e90 con 0x7f73e80ac100 2024-03-16T08:15:40.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.157+0000 7f73f37fe700 1 -- 172.21.15.49:0/2777910909 <== osd.10 v2:172.21.15.129:6824/2725967668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f73fc062e90 con 0x7f73e80ac100 2024-03-16T08:15:40.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] conn(0x7f73e80ac100 msgr2=0x7f73e80ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] conn(0x7f73e80ac100 0x7f73e80ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73e80a2920 msgr2=0x7f73e80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73e80a2920 0x7f73e80a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec005fb0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 msgr2=0x7f73fc0745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 --2- 172.21.15.49:0/2777910909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73fc102510 0x7f73fc0745d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9a0 tx=0x7f73f800ecb0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f7402d8b700 1 -- 172.21.15.49:0/2777910909 reap_dead start 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 shutdown_connections 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 >> 172.21.15.49:0/2777910909 conn(0x7f73fc0fdd80 msgr2=0x7f73fc1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 shutdown_connections 2024-03-16T08:15:40.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.158+0000 7f74047ee700 1 -- 172.21.15.49:0/2777910909 wait complete. 2024-03-16T08:15:40.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2593659160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a94107640 msgr2=0x7f5a9410f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2593659160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a94107640 0x7f5a9410f1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00b0a0 tx=0x7f5a8c00b3b0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2593659160 shutdown_connections 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2593659160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a94107640 0x7f5a9410f1c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2593659160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9406abd0 0x7f5a9406afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2593659160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9406b580 0x7f5a94106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2593659160 >> 172.21.15.49:0/2593659160 conn(0x7f5a94078140 msgr2=0x7f5a94078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2593659160 shutdown_connections 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.807+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2593659160 wait complete. 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 Processor -- start 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 -- start start 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a9406abd0 0x7f5a9419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9406b580 0x7f5a9419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 0x7f5a941988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941125e0 con 0x7f5a94107640 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112460 con 0x7f5a9406abd0 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a9b11d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112760 con 0x7f5a9406b580 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a996ba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 0x7f5a941988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a996ba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 0x7f5a941988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34878/0 (socket says 172.21.15.49:34878) 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a996ba700 1 -- 172.21.15.49:0/2504425675 learned_addr learned my addr 172.21.15.49:0/2504425675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a98eb9700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a9406abd0 0x7f5a9419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.808+0000 7f5a93fff700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9406b580 0x7f5a9419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 -- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9406b580 msgr2=0x7f5a9419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9406b580 0x7f5a9419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 -- 172.21.15.49:0/2504425675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a9406abd0 msgr2=0x7f5a9419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a9406abd0 0x7f5a9419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a94199040 con 0x7f5a94107640 2024-03-16T08:15:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a98eb9700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a9406abd0 0x7f5a9419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a996ba700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 0x7f5a941988e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006070 tx=0x7f5a8c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c021070 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941992d0 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c003c00 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0047a0 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.809+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a55f0 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.810+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a940629e0 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.811+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f5a8c007980 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a91ffb700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a840a2920 0x7f5a840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a98eb9700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a840a2920 0x7f5a840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5a8c00f020 con 0x7f5a94107640 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a91ffb700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] conn(0x7f5a840ac100 0x7f5a840ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a840aebd0 con 0x7f5a840ac100 2024-03-16T08:15:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.812+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5a8c0d0420 con 0x7f5a94107640 2024-03-16T08:15:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.813+0000 7f5a93fff700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] conn(0x7f5a840ac100 0x7f5a840ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.813+0000 7f5a98eb9700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a840a2920 0x7f5a840a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a94199a20 tx=0x7f5a88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.814+0000 7f5a93fff700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] conn(0x7f5a840ac100 0x7f5a840ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.815+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== osd.8 v2:172.21.15.154:6816/160187571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a840aebd0 con 0x7f5a840ac100 2024-03-16T08:15:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.832+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 --> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a94062e10 con 0x7f5a840ac100 2024-03-16T08:15:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.832+0000 7f5a91ffb700 1 -- 172.21.15.49:0/2504425675 <== osd.8 v2:172.21.15.154:6816/160187571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5a94062e10 con 0x7f5a840ac100 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.832+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] conn(0x7f5a840ac100 msgr2=0x7f5a840ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.832+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] conn(0x7f5a840ac100 0x7f5a840ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a840a2920 msgr2=0x7f5a840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f5a840a2920 0x7f5a840a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a94199a20 tx=0x7f5a88009260 comp rx=0 tx=0).stop 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 msgr2=0x7f5a941988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 --2- 172.21.15.49:0/2504425675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a94107640 0x7f5a941988e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006070 tx=0x7f5a8c004d00 comp rx=0 tx=0).stop 2024-03-16T08:15:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a996ba700 1 -- 172.21.15.49:0/2504425675 reap_dead start 2024-03-16T08:15:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 shutdown_connections 2024-03-16T08:15:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 >> 172.21.15.49:0/2504425675 conn(0x7f5a94078140 msgr2=0x7f5a940ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 shutdown_connections 2024-03-16T08:15:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.833+0000 7f5a9b11d700 1 -- 172.21.15.49:0/2504425675 wait complete. 2024-03-16T08:15:41.143 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:41 smithi049 ceph-mon[31942]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:41 smithi129 ceph-mon[36561]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:41 smithi154 ceph-mon[36471]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:41.628 INFO:teuthology.orchestra.run.smithi049.stdout:313532612633 2024-03-16T08:15:41.629 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-16T08:15:41.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.581+0000 7fef70df0700 1 -- 172.21.15.49:0/2902345525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c069f80 msgr2=0x7fef6c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.581+0000 7fef70df0700 1 --2- 172.21.15.49:0/2902345525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c069f80 0x7fef6c06a3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fef60009920 tx=0x7fef60009c30 comp rx=0 tx=0).stop 2024-03-16T08:15:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 -- 172.21.15.49:0/2902345525 shutdown_connections 2024-03-16T08:15:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 --2- 172.21.15.49:0/2902345525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 --2- 172.21.15.49:0/2902345525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef6c108320 0x7fef6c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 --2- 172.21.15.49:0/2902345525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c069f80 0x7fef6c06a3e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 -- 172.21.15.49:0/2902345525 >> 172.21.15.49:0/2902345525 conn(0x7fef6c078100 msgr2=0x7fef6c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 -- 172.21.15.49:0/2902345525 shutdown_connections 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.582+0000 7fef70df0700 1 -- 172.21.15.49:0/2902345525 wait complete. 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 Processor -- start 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 -- start start 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef6c069f80 0x7fef6c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c108320 0x7fef6c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c112580 con 0x7fef6c06a920 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c112400 con 0x7fef6c069f80 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef70df0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c112700 con 0x7fef6c108320 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34866/0 (socket says 172.21.15.49:34866) 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 -- 172.21.15.49:0/2635562276 learned_addr learned my addr 172.21.15.49:0/2635562276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef6a59c700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef6c069f80 0x7fef6c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef6ad9d700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c108320 0x7fef6c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 -- 172.21.15.49:0/2635562276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c108320 msgr2=0x7fef6c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef6c108320 0x7fef6c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 -- 172.21.15.49:0/2635562276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef6c069f80 msgr2=0x7fef6c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef6c069f80 0x7fef6c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.583+0000 7fef69d9b700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c199060 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef69d9b700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c19ec80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e90 tx=0x7fef6000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600050c0 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1992f0 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6000b560 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60003af0 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.584+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a5610 con 0x7fef6c06a920 2024-03-16T08:15:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.585+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef6c062a60 con 0x7fef6c06a920 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.586+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fef6000baa0 con 0x7fef6c06a920 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef5b7fe700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef540a29a0 0x7fef540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef6a59c700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef540a29a0 0x7fef540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fef600cfce0 con 0x7fef6c06a920 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef5b7fe700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] conn(0x7fef540ac050 0x7fef540ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef60001d30 con 0x7fef540ac050 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fef600d01c0 con 0x7fef6c06a920 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef6ad9d700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] conn(0x7fef540ac050 0x7fef540ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.587+0000 7fef6a59c700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef540a29a0 0x7fef540a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef6c199a40 tx=0x7fef5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.588+0000 7fef6ad9d700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] conn(0x7fef540ac050 0x7fef540ae470 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.588+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== osd.0 v2:172.21.15.49:6802/3756414183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef60001d30 con 0x7fef540ac050 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 --> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef6c062e60 con 0x7fef540ac050 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef5b7fe700 1 -- 172.21.15.49:0/2635562276 <== osd.0 v2:172.21.15.49:6802/3756414183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef6c062e60 con 0x7fef540ac050 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] conn(0x7fef540ac050 msgr2=0x7fef540ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] conn(0x7fef540ac050 0x7fef540ae470 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef540a29a0 msgr2=0x7fef540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fef540a29a0 0x7fef540a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef6c199a40 tx=0x7fef5c009260 comp rx=0 tx=0).stop 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 msgr2=0x7fef6c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 --2- 172.21.15.49:0/2635562276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef6c06a920 0x7fef6c19ec80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e90 tx=0x7fef6000fa90 comp rx=0 tx=0).stop 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef6ad9d700 1 -- 172.21.15.49:0/2635562276 reap_dead start 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 shutdown_connections 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.605+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 >> 172.21.15.49:0/2635562276 conn(0x7fef6c078100 msgr2=0x7fef6c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.606+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 shutdown_connections 2024-03-16T08:15:41.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:40.606+0000 7fef70df0700 1 -- 172.21.15.49:0/2635562276 wait complete. 2024-03-16T08:15:42.383 INFO:teuthology.orchestra.run.smithi049.stdout:34359738444 2024-03-16T08:15:42.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-16T08:15:42.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 -- 172.21.15.49:0/1291539885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 msgr2=0x7fe9b8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/1291539885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b8108780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009990 tx=0x7fe9a8009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 -- 172.21.15.49:0/1291539885 shutdown_connections 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/1291539885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102d10 0x7fe9b810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/1291539885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b8108780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/1291539885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b8102350 0x7fe9b81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 -- 172.21.15.49:0/1291539885 >> 172.21.15.49:0/1291539885 conn(0x7fe9b8079390 msgr2=0x7fe9b80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 -- 172.21.15.49:0/1291539885 shutdown_connections 2024-03-16T08:15:42.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.609+0000 7fe9bcd42700 1 -- 172.21.15.49:0/1291539885 wait complete. 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.610+0000 7fe9bcd42700 1 Processor -- start 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.610+0000 7fe9bcd42700 1 -- start start 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.610+0000 7fe9bcd42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 0x7fe9b80739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.610+0000 7fe9bcd42700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b8102d10 0x7fe9b8073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.610+0000 7fe9bcd42700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9bcd42700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8112580 con 0x7fe9b81083a0 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9bcd42700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8112400 con 0x7fe9b8102d10 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9bcd42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8112700 con 0x7fe9b8102350 2024-03-16T08:15:42.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 0x7fe9b80739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48168/0 (socket says 172.21.15.49:48168) 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b6d9d700 1 -- 172.21.15.49:0/783792320 learned_addr learned my addr 172.21.15.49:0/783792320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 0x7fe9b80739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:33680/0 (socket says 172.21.15.49:33680) 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b5d9b700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b8102d10 0x7fe9b8073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 -- 172.21.15.49:0/783792320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b8102d10 msgr2=0x7fe9b8073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b8102d10 0x7fe9b8073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 -- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 msgr2=0x7fe9b806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b81083a0 0x7fe9b806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b806d4e0 con 0x7fe9b8102350 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9b659c700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 0x7fe9b80739e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80059d0 tx=0x7fe9a800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8027070 con 0x7fe9b8102350 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a8003d70 con 0x7fe9b8102350 2024-03-16T08:15:42.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.611+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022e30 con 0x7fe9b8102350 2024-03-16T08:15:42.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.612+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b806d770 con 0x7fe9b8102350 2024-03-16T08:15:42.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.612+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b80716e0 con 0x7fe9b8102350 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.613+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9b8071c10 con 0x7fe9b8102350 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.614+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe9a802a070 con 0x7fe9b8102350 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.614+0000 7fe9a77fe700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe9a00a29a0 0x7fe9a00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.614+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe9a80cf9b0 con 0x7fe9b8102350 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.614+0000 7fe9b5d9b700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe9a00a29a0 0x7fe9a00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9a77fe700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] conn(0x7fe9a00ac180 0x7fe9a00ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9a00aec50 con 0x7fe9a00ac180 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe9a80cfd60 con 0x7fe9b8102350 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9b6d9d700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] conn(0x7fe9a00ac180 0x7fe9a00ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9b5d9b700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe9a00a29a0 0x7fe9a00a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806e230 tx=0x7fe9ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.615+0000 7fe9b6d9d700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] conn(0x7fe9a00ac180 0x7fe9a00ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.616+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== osd.5 v2:172.21.15.154:6808/4188101354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9a00aec50 con 0x7fe9a00ac180 2024-03-16T08:15:42.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.638+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 --> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9b8072060 con 0x7fe9a00ac180 2024-03-16T08:15:42.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9a77fe700 1 -- 172.21.15.49:0/783792320 <== osd.5 v2:172.21.15.154:6808/4188101354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9b8072060 con 0x7fe9a00ac180 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] conn(0x7fe9a00ac180 msgr2=0x7fe9a00ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] conn(0x7fe9a00ac180 0x7fe9a00ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe9a00a29a0 msgr2=0x7fe9a00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe9a00a29a0 0x7fe9a00a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806e230 tx=0x7fe9ac009260 comp rx=0 tx=0).stop 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 msgr2=0x7fe9b80739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 --2- 172.21.15.49:0/783792320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9b8102350 0x7fe9b80739e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80059d0 tx=0x7fe9a800bab0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9b6d9d700 1 -- 172.21.15.49:0/783792320 reap_dead start 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 shutdown_connections 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 >> 172.21.15.49:0/783792320 conn(0x7fe9b8079390 msgr2=0x7fe9b81010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.639+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 shutdown_connections 2024-03-16T08:15:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.640+0000 7fe9bcd42700 1 -- 172.21.15.49:0/783792320 wait complete. 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 -- 172.21.15.49:0/2053829551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 msgr2=0x7f2080102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 --2- 172.21.15.49:0/2053829551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f2080102990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20700099c0 tx=0x7f2070009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 -- 172.21.15.49:0/2053829551 shutdown_connections 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 --2- 172.21.15.49:0/2053829551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2080107660 0x7f208010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 --2- 172.21.15.49:0/2053829551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f2080102990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 --2- 172.21.15.49:0/2053829551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080102f60 0x7f2080106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 -- 172.21.15.49:0/2053829551 >> 172.21.15.49:0/2053829551 conn(0x7f20800793f0 msgr2=0x7f2080079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 -- 172.21.15.49:0/2053829551 shutdown_connections 2024-03-16T08:15:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.951+0000 7f208494b700 1 -- 172.21.15.49:0/2053829551 wait complete. 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 Processor -- start 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 -- start start 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f20800745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080102f60 0x7f2080074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2080107660 0x7f2080075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2080112610 con 0x7f20801025b0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2080112490 con 0x7f2080107660 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.952+0000 7f208494b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2080112790 con 0x7f2080102f60 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f20800745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f20800745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48198/0 (socket says 172.21.15.49:48198) 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 -- 172.21.15.49:0/2851424871 learned_addr learned my addr 172.21.15.49:0/2851424871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207dd9b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080102f60 0x7f2080074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207ed9d700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2080107660 0x7f2080075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 -- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080102f60 msgr2=0x7f2080074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080102f60 0x7f2080074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 -- 172.21.15.49:0/2851424871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2080107660 msgr2=0x7f2080075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2080107660 0x7f2080075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a6fa0 con 0x7f20801025b0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f207e59c700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f20800745d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2070006010 tx=0x7f207000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070003680 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20700050d0 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070003db0 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a7230 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.953+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a7630 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.954+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20800629e0 con 0x7f20801025b0 2024-03-16T08:15:42.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.956+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2070005280 con 0x7f20801025b0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.956+0000 7f206f7fe700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f20680a2ac0 0x7f20680a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f20700cfaa0 con 0x7f20801025b0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f207dd9b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f20680a2ac0 0x7f20680a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f206f7fe700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] conn(0x7f20680ac2a0 0x7f20680ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2070003680 con 0x7f20680ac2a0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f20700cff80 con 0x7f20801025b0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f207dd9b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f20680a2ac0 0x7f20680a4f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2080108ad0 tx=0x7f2074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f207ed9d700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] conn(0x7f20680ac2a0 0x7f20680ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:42.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.957+0000 7f207ed9d700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] conn(0x7f20680ac2a0 0x7f20680ae6c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:42.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.958+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== osd.11 v2:172.21.15.154:6824/901406241 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2070003680 con 0x7f20680ac2a0 2024-03-16T08:15:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.976+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 --> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2080062e60 con 0x7f20680ac2a0 2024-03-16T08:15:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.976+0000 7f206f7fe700 1 -- 172.21.15.49:0/2851424871 <== osd.11 v2:172.21.15.154:6824/901406241 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2080062e60 con 0x7f20680ac2a0 2024-03-16T08:15:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] conn(0x7f20680ac2a0 msgr2=0x7f20680ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] conn(0x7f20680ac2a0 0x7f20680ae6c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f20680a2ac0 msgr2=0x7f20680a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f20680a2ac0 0x7f20680a4f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2080108ad0 tx=0x7f2074009260 comp rx=0 tx=0).stop 2024-03-16T08:15:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 msgr2=0x7f20800745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 --2- 172.21.15.49:0/2851424871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20801025b0 0x7f20800745d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2070006010 tx=0x7f207000f690 comp rx=0 tx=0).stop 2024-03-16T08:15:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f207ed9d700 1 -- 172.21.15.49:0/2851424871 reap_dead start 2024-03-16T08:15:42.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 shutdown_connections 2024-03-16T08:15:42.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 >> 172.21.15.49:0/2851424871 conn(0x7f20800793f0 msgr2=0x7f2080101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:42.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 shutdown_connections 2024-03-16T08:15:42.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:42.977+0000 7f208494b700 1 -- 172.21.15.49:0/2851424871 wait complete. 2024-03-16T08:15:43.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:43 smithi049 ceph-mon[31942]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:43 smithi129 ceph-mon[36561]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:43.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:43 smithi154 ceph-mon[36471]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:43.782 INFO:teuthology.orchestra.run.smithi049.stdout:167503724596 2024-03-16T08:15:43.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-16T08:15:44.397 INFO:teuthology.orchestra.run.smithi049.stdout:257698037795 2024-03-16T08:15:44.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-16T08:15:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:45 smithi049 ceph-mon[31942]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T08:15:45.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:45 smithi129 ceph-mon[36561]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 -- 172.21.15.49:0/1569907613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61901083a0 msgr2=0x7f6190108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 --2- 172.21.15.49:0/1569907613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61901083a0 0x7f6190108780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f618c009990 tx=0x7f618c009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 -- 172.21.15.49:0/1569907613 shutdown_connections 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 --2- 172.21.15.49:0/1569907613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 0x7f619010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 --2- 172.21.15.49:0/1569907613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61901083a0 0x7f6190108780 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 --2- 172.21.15.49:0/1569907613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190102350 0x7f61901027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.469+0000 7f61985d3700 1 -- 172.21.15.49:0/1569907613 >> 172.21.15.49:0/1569907613 conn(0x7f6190079390 msgr2=0x7f61900797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.470+0000 7f61985d3700 1 -- 172.21.15.49:0/1569907613 shutdown_connections 2024-03-16T08:15:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.470+0000 7f61985d3700 1 -- 172.21.15.49:0/1569907613 wait complete. 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.470+0000 7f61985d3700 1 Processor -- start 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.470+0000 7f61985d3700 1 -- start start 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 0x7f619019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 0x7f619019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 0x7f61901987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112580 con 0x7f6190102350 2024-03-16T08:15:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112400 con 0x7f61901083a0 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f61985d3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112700 con 0x7f6190102d10 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f619636f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 0x7f619019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6196b70700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 0x7f61901987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6196b70700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 0x7f61901987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:43410/0 (socket says 172.21.15.49:43410) 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f619636f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 0x7f619019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48222/0 (socket says 172.21.15.49:48222) 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 0x7f619019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6196b70700 1 -- 172.21.15.49:0/3901954172 learned_addr learned my addr 172.21.15.49:0/3901954172 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 -- 172.21.15.49:0/3901954172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 msgr2=0x7f61901987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 0x7f61901987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 -- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 msgr2=0x7f619019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 0x7f619019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6195b6e700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190198fc0 con 0x7f6190102d10 2024-03-16T08:15:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f6196b70700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61901083a0 0x7f61901987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.471+0000 7f619636f700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190102350 0x7f619019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f6195b6e700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 0x7f619019eba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f618000e9f0 tx=0x7f618000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618000c9c0 con 0x7f6190102d10 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618000cb60 con 0x7f6190102d10 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180018680 con 0x7f6190102d10 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61901992b0 con 0x7f6190102d10 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.472+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901a54c0 con 0x7f6190102d10 2024-03-16T08:15:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.473+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6190062a60 con 0x7f6190102d10 2024-03-16T08:15:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.475+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6180010430 con 0x7f6190102d10 2024-03-16T08:15:45.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:45 smithi154 ceph-mon[36471]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.475+0000 7f61877fe700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f617c0a29a0 0x7f617c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.475+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f61800cf050 con 0x7f6190102d10 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.475+0000 7f619636f700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f617c0a29a0 0x7f617c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f61877fe700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] conn(0x7f617c0ac180 0x7f617c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f618001bec0 con 0x7f617c0ac180 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f61800cf400 con 0x7f6190102d10 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f6196b70700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] conn(0x7f617c0ac180 0x7f617c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f619636f700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f617c0a29a0 0x7f617c0a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b2e0 tx=0x7f618c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.476+0000 7f6196b70700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] conn(0x7f617c0ac180 0x7f617c0ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.477+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== osd.6 v2:172.21.15.49:6818/3010410053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f618001bec0 con 0x7f617c0ac180 2024-03-16T08:15:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.496+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 --> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6190062e60 con 0x7f617c0ac180 2024-03-16T08:15:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.496+0000 7f61877fe700 1 -- 172.21.15.49:0/3901954172 <== osd.6 v2:172.21.15.49:6818/3010410053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6190062e60 con 0x7f617c0ac180 2024-03-16T08:15:45.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.496+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] conn(0x7f617c0ac180 msgr2=0x7f617c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.496+0000 7f61985d3700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] conn(0x7f617c0ac180 0x7f617c0ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:45.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f617c0a29a0 msgr2=0x7f617c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f617c0a29a0 0x7f617c0a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b2e0 tx=0x7f618c005960 comp rx=0 tx=0).stop 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 msgr2=0x7f619019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 --2- 172.21.15.49:0/3901954172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6190102d10 0x7f619019eba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f618000e9f0 tx=0x7f618000ed00 comp rx=0 tx=0).stop 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f6196b70700 1 -- 172.21.15.49:0/3901954172 reap_dead start 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 shutdown_connections 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 >> 172.21.15.49:0/3901954172 conn(0x7f6190079390 msgr2=0x7f61901010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 shutdown_connections 2024-03-16T08:15:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:45.497+0000 7f61985d3700 1 -- 172.21.15.49:0/3901954172 wait complete. 2024-03-16T08:15:46.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 -- 172.21.15.49:0/3445766313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f316006a920 msgr2=0x7f316010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 --2- 172.21.15.49:0/3445766313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f316006a920 0x7f316010f160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f315800b0d0 tx=0x7f315800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 -- 172.21.15.49:0/3445766313 shutdown_connections 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 --2- 172.21.15.49:0/3445766313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f316006a920 0x7f316010f160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 --2- 172.21.15.49:0/3445766313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3160108320 0x7f3160108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 --2- 172.21.15.49:0/3445766313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3160069f80 0x7f316006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 -- 172.21.15.49:0/3445766313 >> 172.21.15.49:0/3445766313 conn(0x7f3160078100 msgr2=0x7f3160078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 -- 172.21.15.49:0/3445766313 shutdown_connections 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.313+0000 7f31687dc700 1 -- 172.21.15.49:0/3445766313 wait complete. 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 Processor -- start 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 -- start start 2024-03-16T08:15:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3160069f80 0x7f31601012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316006a920 0x7f31601017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 0x7f3160101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31601125a0 con 0x7f3160108320 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3160112420 con 0x7f316006a920 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f31687dc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3160112720 con 0x7f3160069f80 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 0x7f3160101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166578700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3160069f80 0x7f31601012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 0x7f3160101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48236/0 (socket says 172.21.15.49:48236) 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 -- 172.21.15.49:0/3884963013 learned_addr learned my addr 172.21.15.49:0/3884963013 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3165d77700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316006a920 0x7f31601017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 -- 172.21.15.49:0/3884963013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3160069f80 msgr2=0x7f31601012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3160069f80 0x7f31601012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 -- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316006a920 msgr2=0x7f31601017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316006a920 0x7f31601017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3166d79700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31601a6f30 con 0x7f3160108320 2024-03-16T08:15:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.314+0000 7f3165d77700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316006a920 0x7f31601017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f3166d79700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 0x7f3160101d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3158006070 tx=0x7f3158007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158003c00 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3158022070 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31601a71c0 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315801b6c0 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.315+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31601a75c0 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.316+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31600629e0 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.317+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f315801d470 con 0x7f3160108320 2024-03-16T08:15:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f31577fe700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f314c0a2920 0x7f314c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f3166578700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f314c0a2920 0x7f314c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31580d0400 con 0x7f3160108320 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f31577fe700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] conn(0x7f314c0abfd0 0x7f314c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f314c0aeaa0 con 0x7f314c0abfd0 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f31580d08e0 con 0x7f3160108320 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f3166578700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f314c0a2920 0x7f314c0a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f315c009be0 tx=0x7f315c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.318+0000 7f3165d77700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] conn(0x7f314c0abfd0 0x7f314c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.319+0000 7f3165d77700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] conn(0x7f314c0abfd0 0x7f314c0ae3f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.319+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== osd.7 v2:172.21.15.129:6816/252652011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f314c0aeaa0 con 0x7f314c0abfd0 2024-03-16T08:15:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.336+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 --> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3160062de0 con 0x7f314c0abfd0 2024-03-16T08:15:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.336+0000 7f31577fe700 1 -- 172.21.15.49:0/3884963013 <== osd.7 v2:172.21.15.129:6816/252652011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3160062de0 con 0x7f314c0abfd0 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] conn(0x7f314c0abfd0 msgr2=0x7f314c0ae3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] conn(0x7f314c0abfd0 0x7f314c0ae3f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f314c0a2920 msgr2=0x7f314c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f314c0a2920 0x7f314c0a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f315c009be0 tx=0x7f315c009260 comp rx=0 tx=0).stop 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 msgr2=0x7f3160101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 --2- 172.21.15.49:0/3884963013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3160108320 0x7f3160101d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3158006070 tx=0x7f3158007a20 comp rx=0 tx=0).stop 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f3166d79700 1 -- 172.21.15.49:0/3884963013 reap_dead start 2024-03-16T08:15:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 shutdown_connections 2024-03-16T08:15:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 >> 172.21.15.49:0/3884963013 conn(0x7f3160078100 msgr2=0x7f31600ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 shutdown_connections 2024-03-16T08:15:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.337+0000 7f31687dc700 1 -- 172.21.15.49:0/3884963013 wait complete. 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 -- 172.21.15.49:0/1213211628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 msgr2=0x7f64dc1086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/1213211628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 0x7f64dc1086e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009920 tx=0x7f64d8009c30 comp rx=0 tx=0).stop 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 -- 172.21.15.49:0/1213211628 shutdown_connections 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/1213211628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102ce0 0x7f64dc10f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/1213211628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 0x7f64dc1086e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/1213211628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102320 0x7f64dc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 -- 172.21.15.49:0/1213211628 >> 172.21.15.49:0/1213211628 conn(0x7f64dc079400 msgr2=0x7f64dc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 -- 172.21.15.49:0/1213211628 shutdown_connections 2024-03-16T08:15:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.920+0000 7f64e3dd6700 1 -- 172.21.15.49:0/1213211628 wait complete. 2024-03-16T08:15:46.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 Processor -- start 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 -- start start 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102320 0x7f64dc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 0x7f64dc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 0x7f64dc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1125d0 con 0x7f64dc102ce0 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc112450 con 0x7f64dc108300 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e3dd6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc112750 con 0x7f64dc102320 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 0x7f64dc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1b72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102320 0x7f64dc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 0x7f64dc19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48252/0 (socket says 172.21.15.49:48252) 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 -- 172.21.15.49:0/3718627709 learned_addr learned my addr 172.21.15.49:0/3718627709 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e2373700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 0x7f64dc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 -- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102320 msgr2=0x7f64dc19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102320 0x7f64dc19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 -- 172.21.15.49:0/3718627709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 msgr2=0x7f64dc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64dc108300 0x7f64dc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1371700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc199050 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.921+0000 7f64e1b72700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64dc102320 0x7f64dc19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64e1371700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 0x7f64dc19ec70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00ec10 tx=0x7f64cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00cc80 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc199340 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64cc004500 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc010690 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.922+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc1a5600 con 0x7f64dc102ce0 2024-03-16T08:15:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.923+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64dc062a60 con 0x7f64dc102ce0 2024-03-16T08:15:46.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.924+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f64cc003e10 con 0x7f64dc102ce0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.924+0000 7f64d2ffd700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64c80a2920 0x7f64c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64e1b72700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64c80a2920 0x7f64c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f64cc0cefd0 con 0x7f64dc102ce0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64d2ffd700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] conn(0x7f64c80ac100 0x7f64c80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64cc012ec0 con 0x7f64c80ac100 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f64cc0cf4b0 con 0x7f64dc102ce0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64e2373700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] conn(0x7f64c80ac100 0x7f64c80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64e1b72700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64c80a2920 0x7f64c80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64dc199a30 tx=0x7f64d8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.925+0000 7f64e2373700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] conn(0x7f64c80ac100 0x7f64c80ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:46.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.927+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== osd.2 v2:172.21.15.154:6800/487578356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64cc012ec0 con 0x7f64c80ac100 2024-03-16T08:15:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.943+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 --> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64dc062e60 con 0x7f64c80ac100 2024-03-16T08:15:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.943+0000 7f64d2ffd700 1 -- 172.21.15.49:0/3718627709 <== osd.2 v2:172.21.15.154:6800/487578356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f64dc062e60 con 0x7f64c80ac100 2024-03-16T08:15:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] conn(0x7f64c80ac100 msgr2=0x7f64c80ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] conn(0x7f64c80ac100 0x7f64c80ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64c80a2920 msgr2=0x7f64c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f64c80a2920 0x7f64c80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64dc199a30 tx=0x7f64d8019040 comp rx=0 tx=0).stop 2024-03-16T08:15:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 msgr2=0x7f64dc19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 --2- 172.21.15.49:0/3718627709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64dc102ce0 0x7f64dc19ec70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00ec10 tx=0x7f64cc00ef20 comp rx=0 tx=0).stop 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e2373700 1 -- 172.21.15.49:0/3718627709 reap_dead start 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 shutdown_connections 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 >> 172.21.15.49:0/3718627709 conn(0x7f64dc079400 msgr2=0x7f64dc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 shutdown_connections 2024-03-16T08:15:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:46.944+0000 7f64e3dd6700 1 -- 172.21.15.49:0/3718627709 wait complete. 2024-03-16T08:15:47.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:47 smithi049 ceph-mon[31942]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:47.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:47 smithi129 ceph-mon[36561]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:47.455 INFO:teuthology.orchestra.run.smithi049.stdout:347892350997 2024-03-16T08:15:47.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-16T08:15:47.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:47 smithi154 ceph-mon[36471]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T08:15:47.900 INFO:teuthology.orchestra.run.smithi049.stdout:201863462959 2024-03-16T08:15:47.900 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 -- 172.21.15.49:0/1784272541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d00108320 msgr2=0x7f9d00108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 --2- 172.21.15.49:0/1784272541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d00108320 0x7f9d00108700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00099c0 tx=0x7f9cf0009cd0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 -- 172.21.15.49:0/1784272541 shutdown_connections 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 --2- 172.21.15.49:0/1784272541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d0006a920 0x7f9d0010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 --2- 172.21.15.49:0/1784272541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d00108320 0x7f9d00108700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 --2- 172.21.15.49:0/1784272541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00069f80 0x7f9d0006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.206+0000 7f9d0604c700 1 -- 172.21.15.49:0/1784272541 >> 172.21.15.49:0/1784272541 conn(0x7f9d00078100 msgr2=0x7f9d00078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.207+0000 7f9d0604c700 1 -- 172.21.15.49:0/1784272541 shutdown_connections 2024-03-16T08:15:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.207+0000 7f9d0604c700 1 -- 172.21.15.49:0/1784272541 wait complete. 2024-03-16T08:15:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 Processor -- start 2024-03-16T08:15:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 -- start start 2024-03-16T08:15:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d00069f80 0x7f9d0019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 0x7f9d0019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00196860 0x7f9d00196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d00110410 con 0x7f9d0006a920 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d00110290 con 0x7f9d00196860 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9d0604c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d00110590 con 0x7f9d00069f80 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.208+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 0x7f9d0019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 0x7f9d0019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48270/0 (socket says 172.21.15.49:48270) 2024-03-16T08:15:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfeffd700 1 -- 172.21.15.49:0/3157768001 learned_addr learned my addr 172.21.15.49:0/3157768001 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfffff700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00196860 0x7f9d00196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d00069f80 0x7f9d0019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 -- 172.21.15.49:0/3157768001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00196860 msgr2=0x7f9d00196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00196860 0x7f9d00196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 -- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 msgr2=0x7f9d0019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 0x7f9d0019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d001973e0 con 0x7f9d00069f80 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfffff700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00196860 0x7f9d00196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfeffd700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d0006a920 0x7f9d0019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:15:48.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cff7fe700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d00069f80 0x7f9d0019c6a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0008000 tx=0x7f9cf000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0026070 con 0x7f9d00069f80 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf0003680 con 0x7f9d00069f80 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.209+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d00197670 con 0x7f9d00069f80 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.210+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00173f0 con 0x7f9d00069f80 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.210+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001a37f0 con 0x7f9d00069f80 2024-03-16T08:15:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.210+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d00062a20 con 0x7f9d00069f80 2024-03-16T08:15:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.212+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9cf0029070 con 0x7f9d00069f80 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.212+0000 7f9cfcff9700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9ce80a2920 0x7f9ce80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfeffd700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9ce80a2920 0x7f9ce80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9cf00d00c0 con 0x7f9d00069f80 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfcff9700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] conn(0x7f9ce80ac100 0x7f9ce80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cf0028e50 con 0x7f9ce80ac100 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9cf00d0470 con 0x7f9d00069f80 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfffff700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] conn(0x7f9ce80ac100 0x7f9ce80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfeffd700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9ce80a2920 0x7f9ce80a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d000789b0 tx=0x7f9cf400b810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.213+0000 7f9cfffff700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] conn(0x7f9ce80ac100 0x7f9ce80ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.214+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== osd.9 v2:172.21.15.49:6826/1471236265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cf0028e50 con 0x7f9ce80ac100 2024-03-16T08:15:48.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 --> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d00062ea0 con 0x7f9ce80ac100 2024-03-16T08:15:48.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9cfcff9700 1 -- 172.21.15.49:0/3157768001 <== osd.9 v2:172.21.15.49:6826/1471236265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9d00062ea0 con 0x7f9ce80ac100 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] conn(0x7f9ce80ac100 msgr2=0x7f9ce80ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] conn(0x7f9ce80ac100 0x7f9ce80ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9ce80a2920 msgr2=0x7f9ce80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9ce80a2920 0x7f9ce80a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d000789b0 tx=0x7f9cf400b810 comp rx=0 tx=0).stop 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d00069f80 msgr2=0x7f9d0019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 --2- 172.21.15.49:0/3157768001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d00069f80 0x7f9d0019c6a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0008000 tx=0x7f9cf000fa00 comp rx=0 tx=0).stop 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9cfffff700 1 -- 172.21.15.49:0/3157768001 reap_dead start 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 shutdown_connections 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.231+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 >> 172.21.15.49:0/3157768001 conn(0x7f9d00078100 msgr2=0x7f9d001001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.232+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 shutdown_connections 2024-03-16T08:15:48.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:48.232+0000 7f9d0604c700 1 -- 172.21.15.49:0/3157768001 wait complete. 2024-03-16T08:15:48.934 INFO:teuthology.orchestra.run.smithi049.stdout:77309411396 2024-03-16T08:15:48.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-16T08:15:49.215 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:49.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:49 smithi049 ceph-mon[31942]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:49.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:49 smithi129 ceph-mon[36561]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:49.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:49 smithi154 ceph-mon[36471]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-16T08:15:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:50.434 INFO:teuthology.orchestra.run.smithi049.stdout:292057776161 2024-03-16T08:15:50.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-16T08:15:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:51.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:51 smithi049 ceph-mon[31942]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-16T08:15:51.139 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:51.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:51 smithi129 ceph-mon[36561]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-16T08:15:51.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:51 smithi154 ceph-mon[36471]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-16T08:15:51.686 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:51.686 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:51.791 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:15:52.202 INFO:teuthology.orchestra.run.smithi049.stdout:223338299434 2024-03-16T08:15:52.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-16T08:15:53.229 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:53 smithi049 ceph-mon[31942]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T08:15:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:53 smithi129 ceph-mon[36561]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T08:15:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:53 smithi154 ceph-mon[36471]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T08:15:55.316 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:55 smithi049 ceph-mon[31942]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-16T08:15:55.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:55 smithi129 ceph-mon[36561]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-16T08:15:55.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:55 smithi154 ceph-mon[36471]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-16T08:15:57.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:57 smithi049 ceph-mon[31942]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:15:57.331 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:57 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2156000193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T08:15:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:57 smithi129 ceph-mon[36561]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:15:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:57 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2156000193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T08:15:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:57 smithi154 ceph-mon[36471]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:15:57.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:57 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2156000193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 -- 172.21.15.49:0/3477973409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c601064b0 msgr2=0x7f4c6010e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 --2- 172.21.15.49:0/3477973409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c601064b0 0x7f4c6010e080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800b0d0 tx=0x7f4c5800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 -- 172.21.15.49:0/3477973409 shutdown_connections 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 --2- 172.21.15.49:0/3477973409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c601064b0 0x7f4c6010e080 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 --2- 172.21.15.49:0/3477973409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101460 0x7f4c60101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 --2- 172.21.15.49:0/3477973409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c60101e10 0x7f4c60105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 -- 172.21.15.49:0/3477973409 >> 172.21.15.49:0/3477973409 conn(0x7f4c60077220 msgr2=0x7f4c60077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 -- 172.21.15.49:0/3477973409 shutdown_connections 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.531+0000 7f4c664f9700 1 -- 172.21.15.49:0/3477973409 wait complete. 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 Processor -- start 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 -- start start 2024-03-16T08:15:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101e10 0x7f4c601033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c601064b0 0x7f4c601038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601114c0 con 0x7f4c60101460 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60111340 con 0x7f4c601064b0 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c664f9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60111640 con 0x7f4c60101e10 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:57.651 INFO:teuthology.orchestra.run.smithi049.stdout:55834574922 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39164/0 (socket says 172.21.15.49:39164) 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5ffff700 1 -- 172.21.15.49:0/2156000193 learned_addr learned my addr 172.21.15.49:0/2156000193 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5f7fe700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101e10 0x7f4c601033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c64a96700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c601064b0 0x7f4c601038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5ffff700 1 -- 172.21.15.49:0/2156000193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101e10 msgr2=0x7f4c601033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.532+0000 7f4c5ffff700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101e10 0x7f4c601033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5ffff700 1 -- 172.21.15.49:0/2156000193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c601064b0 msgr2=0x7f4c601038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5ffff700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c601064b0 0x7f4c601038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5ffff700 1 -- 172.21.15.49:0/2156000193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c601a6f60 con 0x7f4c60101460 2024-03-16T08:15:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5ffff700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9c0 tx=0x7f4c5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000cbd0 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5000ee60 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c601a7250 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50005630 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.533+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a7740 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.534+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c60068f90 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.535+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4c50004750 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.536+0000 7f4c5d7fa700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 0x7f4c480a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.536+0000 7f4c5f7fe700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 0x7f4c480a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.536+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4c500cecc0 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.536+0000 7f4c5f7fe700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 0x7f4c480a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4c60107950 tx=0x7f4c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.541+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4c5009bdb0 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.749+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4c601043a0 con 0x7f4c60101460 2024-03-16T08:15:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.749+0000 7f4c5d7fa700 1 -- 172.21.15.49:0/2156000193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c500bb030 con 0x7f4c60101460 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.751+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 msgr2=0x7f4c480a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.751+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 0x7f4c480a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4c60107950 tx=0x7f4c54009260 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.751+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 msgr2=0x7f4c60102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.751+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9c0 tx=0x7f4c5000ecd0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 shutdown_connections 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4c480a2920 0x7f4c480a4de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c601064b0 0x7f4c601038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c60101460 0x7f4c60102e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 --2- 172.21.15.49:0/2156000193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c60101e10 0x7f4c601033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 >> 172.21.15.49:0/2156000193 conn(0x7f4c60077220 msgr2=0x7f4c6006aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 shutdown_connections 2024-03-16T08:15:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:56.752+0000 7f4c664f9700 1 -- 172.21.15.49:0/2156000193 wait complete. 2024-03-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 -- 172.21.15.49:0/1237787358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8102320 msgr2=0x7f71e81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/1237787358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8102320 0x7f71e81027a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009990 tx=0x7f71d0009ca0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 -- 172.21.15.49:0/1237787358 shutdown_connections 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/1237787358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102ce0 0x7f71e810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/1237787358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8108370 0x7f71e8108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/1237787358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8102320 0x7f71e81027a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 -- 172.21.15.49:0/1237787358 >> 172.21.15.49:0/1237787358 conn(0x7f71e8079390 msgr2=0x7f71e80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 -- 172.21.15.49:0/1237787358 shutdown_connections 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 -- 172.21.15.49:0/1237787358 wait complete. 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.649+0000 7f71ed2e2700 1 Processor -- start 2024-03-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 -- start start 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 0x7f71e819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8108370 0x7f71e8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8112570 con 0x7f71e8108370 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81123f0 con 0x7f71e8102ce0 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71ed2e2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81126f0 con 0x7f71e8102320 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8108370 0x7f71e8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e67fc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 0x7f71e819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:39224/0 (socket says 172.21.15.49:39224) 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e67fc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 0x7f71e819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47314/0 (socket says 172.21.15.49:47314) 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 -- 172.21.15.49:0/2931160754 learned_addr learned my addr 172.21.15.49:0/2931160754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:15:58.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 -- 172.21.15.49:0/2931160754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 msgr2=0x7f71e819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 0x7f71e819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 -- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8108370 msgr2=0x7f71e8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8108370 0x7f71e8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.650+0000 7f71e6ffd700 1 -- 172.21.15.49:0/2931160754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e8199030 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71e6ffd700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e9a0 tx=0x7f71d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d800ca10 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d800cbb0 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e8199320 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8018680 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.651+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e8074500 con 0x7f71e8102320 2024-03-16T08:15:58.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.652+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e81039e0 con 0x7f71e8102320 2024-03-16T08:15:58.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.653+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f71d8047a90 con 0x7f71e8102320 2024-03-16T08:15:58.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.653+0000 7f71dffff700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 0x7f71d40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:15:58.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.654+0000 7f71e67fc700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 0x7f71d40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:15:58.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.654+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f71d80cec10 con 0x7f71e8102320 2024-03-16T08:15:58.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.654+0000 7f71e67fc700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 0x7f71d40a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b2e0 tx=0x7f71d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:15:58.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.657+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f71d809bd00 con 0x7f71e8102320 2024-03-16T08:15:58.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.864+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f71e802db70 con 0x7f71e8102320 2024-03-16T08:15:58.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.865+0000 7f71dffff700 1 -- 172.21.15.49:0/2931160754 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f71d809b600 con 0x7f71e8102320 2024-03-16T08:15:58.866 INFO:teuthology.orchestra.run.smithi049.stdout:257698037799 2024-03-16T08:15:58.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 msgr2=0x7f71d40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:58.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 0x7f71d40a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b2e0 tx=0x7f71d0005960 comp rx=0 tx=0).stop 2024-03-16T08:15:58.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 msgr2=0x7f71e819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:15:58.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e9a0 tx=0x7f71d800ecb0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 shutdown_connections 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f71d40a2920 0x7f71d40a4de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8108370 0x7f71e8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71e8102320 0x7f71e819e680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 --2- 172.21.15.49:0/2931160754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8102ce0 0x7f71e819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 >> 172.21.15.49:0/2931160754 conn(0x7f71e8079390 msgr2=0x7f71e8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 shutdown_connections 2024-03-16T08:15:58.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:15:58.867+0000 7f71ed2e2700 1 -- 172.21.15.49:0/2931160754 wait complete. 2024-03-16T08:15:59.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:59 smithi049 ceph-mon[31942]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:15:59.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:15:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2931160754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T08:15:59.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:59 smithi129 ceph-mon[36561]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:15:59.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:15:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2931160754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T08:15:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:59 smithi154 ceph-mon[36471]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:15:59.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:15:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2931160754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- 172.21.15.49:0/4090733267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 msgr2=0x7ffb84108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/4090733267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84108700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb800099c0 tx=0x7ffb80009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- 172.21.15.49:0/4090733267 shutdown_connections 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/4090733267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 0x7ffb8410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/4090733267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84108700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/4090733267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb8406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- 172.21.15.49:0/4090733267 >> 172.21.15.49:0/4090733267 conn(0x7ffb84078100 msgr2=0x7ffb84078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- 172.21.15.49:0/4090733267 shutdown_connections 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- 172.21.15.49:0/4090733267 wait complete. 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 Processor -- start 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.050+0000 7ffb8bfde700 1 -- start start 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb841012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 0x7ffb841017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84112560 con 0x7ffb84108320 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841123e0 con 0x7ffb84069f80 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8bfde700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841126e0 con 0x7ffb8406a920 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8a57b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb841012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8a57b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39184/0 (socket says 172.21.15.49:39184) 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8a57b700 1 -- 172.21.15.49:0/2487241379 learned_addr learned my addr 172.21.15.49:0/2487241379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89579700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 0x7ffb841017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 -- 172.21.15.49:0/2487241379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 msgr2=0x7ffb841017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 0x7ffb841017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 -- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 msgr2=0x7ffb84101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 -- 172.21.15.49:0/2487241379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb841a6f20 con 0x7ffb84069f80 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb8a57b700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.051+0000 7ffb89d7a700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb841012a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098e0 tx=0x7ffb80005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80027070 con 0x7ffb84069f80 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80003680 con 0x7ffb84069f80 2024-03-16T08:16:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb841a71b0 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80022e10 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb841a7600 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.052+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb84068f90 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.054+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ffb8002a070 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.054+0000 7ffb7affd700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 0x7ffb700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.054+0000 7ffb89579700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 0x7ffb700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.054+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffb800cf9b0 con 0x7ffb84069f80 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.055+0000 7ffb89579700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 0x7ffb700a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb84078670 tx=0x7ffb74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.058+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffb80013070 con 0x7ffb84069f80 2024-03-16T08:16:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.266+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffb8402dd30 con 0x7ffb84069f80 2024-03-16T08:16:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.267+0000 7ffb7affd700 1 -- 172.21.15.49:0/2487241379 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb8009cb50 con 0x7ffb84069f80 2024-03-16T08:16:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:34359738447 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 msgr2=0x7ffb700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 0x7ffb700a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb84078670 tx=0x7ffb74009260 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 msgr2=0x7ffb841012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb841012a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098e0 tx=0x7ffb80005f10 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 shutdown_connections 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffb700a2920 0x7ffb700a4de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb84108320 0x7ffb84101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb84069f80 0x7ffb841012a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 --2- 172.21.15.49:0/2487241379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8406a920 0x7ffb841017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 >> 172.21.15.49:0/2487241379 conn(0x7ffb84078100 msgr2=0x7ffb840ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 shutdown_connections 2024-03-16T08:16:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.269+0000 7ffb8bfde700 1 -- 172.21.15.49:0/2487241379 wait complete. 2024-03-16T08:16:00.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574922 for osd.1 2024-03-16T08:16:00.819 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 -- 172.21.15.49:0/3551228228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 msgr2=0x7eff54106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/3551228228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff54106f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff500099c0 tx=0x7eff50009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 -- 172.21.15.49:0/3551228228 shutdown_connections 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/3551228228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54107650 0x7eff5410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/3551228228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff5406ac90 0x7eff5406b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/3551228228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff54106f20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 -- 172.21.15.49:0/3551228228 >> 172.21.15.49:0/3551228228 conn(0x7eff54077210 msgr2=0x7eff54077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 -- 172.21.15.49:0/3551228228 shutdown_connections 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.795+0000 7eff5b6f0700 1 -- 172.21.15.49:0/3551228228 wait complete. 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 Processor -- start 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 -- start start 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 0x7eff5419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff54107650 0x7eff54198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff541125a0 con 0x7eff5406b640 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff54112420 con 0x7eff54107650 2024-03-16T08:16:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.796+0000 7eff5b6f0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff54112720 con 0x7eff5406ac90 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff58c8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 0x7eff5419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff58c8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39202/0 (socket says 172.21.15.49:39202) 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff58c8b700 1 -- 172.21.15.49:0/2695241419 learned_addr learned my addr 172.21.15.49:0/2695241419 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff59c8d700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff54107650 0x7eff54198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 -- 172.21.15.49:0/2695241419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff54107650 msgr2=0x7eff54198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff54107650 0x7eff54198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 -- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 msgr2=0x7eff5419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 -- 172.21.15.49:0/2695241419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff54199060 con 0x7eff5406ac90 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff58c8b700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5948c700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 0x7eff5419e740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400c9c0 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4400cb60 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff54199350 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44018680 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.797+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541a5610 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.799+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff54068f90 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.800+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7eff44010430 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.800+0000 7eff4a7fc700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 0x7eff400a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.800+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7eff440cebd0 con 0x7eff5406ac90 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.800+0000 7eff58c8b700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 0x7eff400a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.801+0000 7eff58c8b700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 0x7eff400a4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff54100e60 tx=0x7eff50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:00.805+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff4409bbc0 con 0x7eff5406ac90 2024-03-16T08:16:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.012+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7eff5402db70 con 0x7eff5406ac90 2024-03-16T08:16:01.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.013+0000 7eff4a7fc700 1 -- 172.21.15.49:0/2695241419 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff4409b4c0 con 0x7eff5406ac90 2024-03-16T08:16:01.014 INFO:teuthology.orchestra.run.smithi049.stdout:167503724599 2024-03-16T08:16:01.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 msgr2=0x7eff400a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:01.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 0x7eff400a4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff54100e60 tx=0x7eff50005fb0 comp rx=0 tx=0).stop 2024-03-16T08:16:01.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 msgr2=0x7eff5419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:01.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 0x7eff5419e740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400ed00 comp rx=0 tx=0).stop 2024-03-16T08:16:01.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 shutdown_connections 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7eff400a29a0 0x7eff400a4e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff54107650 0x7eff54198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff5406ac90 0x7eff5419e740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 --2- 172.21.15.49:0/2695241419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff5406b640 0x7eff5419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 >> 172.21.15.49:0/2695241419 conn(0x7eff54077210 msgr2=0x7eff540ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 shutdown_connections 2024-03-16T08:16:01.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:01.015+0000 7eff5b6f0700 1 -- 172.21.15.49:0/2695241419 wait complete. 2024-03-16T08:16:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:01 smithi049 ceph-mon[31942]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2487241379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T08:16:01.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2695241419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T08:16:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:01 smithi129 ceph-mon[36561]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2487241379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T08:16:01.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2695241419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T08:16:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:01 smithi154 ceph-mon[36471]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2487241379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T08:16:01.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2695241419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T08:16:01.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037799 for osd.8 2024-03-16T08:16:01.538 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:02 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:16:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:02 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:16:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:02 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:16:02.947 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:02.948 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:03.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 -- 172.21.15.49:0/3415372380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 msgr2=0x7f36d406ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 --2- 172.21.15.49:0/3415372380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d406ae50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36d00099c0 tx=0x7f36d0009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 -- 172.21.15.49:0/3415372380 shutdown_connections 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 --2- 172.21.15.49:0/3415372380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 0x7f36d410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 --2- 172.21.15.49:0/3415372380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d406ae50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 --2- 172.21.15.49:0/3415372380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 0x7f36d4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 -- 172.21.15.49:0/3415372380 >> 172.21.15.49:0/3415372380 conn(0x7f36d4077220 msgr2=0x7f36d4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 -- 172.21.15.49:0/3415372380 shutdown_connections 2024-03-16T08:16:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.147+0000 7f36db624700 1 -- 172.21.15.49:0/3415372380 wait complete. 2024-03-16T08:16:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 Processor -- start 2024-03-16T08:16:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 -- start start 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 0x7f36d4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 0x7f36d4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4112600 con 0x7f36d406aa70 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4112480 con 0x7f36d4107650 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.148+0000 7f36db624700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4112780 con 0x7f36d406b420 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39364/0 (socket says 172.21.15.49:39364) 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 -- 172.21.15.49:0/1156749779 learned_addr learned my addr 172.21.15.49:0/1156749779 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d8bbf700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 0x7f36d4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d9bc1700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 0x7f36d4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 -- 172.21.15.49:0/1156749779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 msgr2=0x7f36d4101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 0x7f36d4101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 -- 172.21.15.49:0/1156749779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 msgr2=0x7f36d4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 0x7f36d4101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 -- 172.21.15.49:0/1156749779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41a7010 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36d93c0700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006010 tx=0x7f36d0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0026070 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a72a0 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d0004cd0 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.149+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0003ba0 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.150+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a7700 con 0x7f36d406aa70 2024-03-16T08:16:03.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.151+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d410a800 con 0x7f36d406aa70 2024-03-16T08:16:03.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.153+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f36d0029070 con 0x7f36d406aa70 2024-03-16T08:16:03.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.154+0000 7f36ca7fc700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 0x7f36c00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.154+0000 7f36d8bbf700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 0x7f36c00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.154+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f36d00cf430 con 0x7f36d406aa70 2024-03-16T08:16:03.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.154+0000 7f36d8bbf700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 0x7f36c00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36d4100d50 tx=0x7f36c400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:03.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.161+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f36d005d740 con 0x7f36d406aa70 2024-03-16T08:16:03.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:03 smithi049 ceph-mon[31942]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:03 smithi129 ceph-mon[36561]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.373+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f36d410a1e0 con 0x7f36d406aa70 2024-03-16T08:16:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.376+0000 7f36ca7fc700 1 -- 172.21.15.49:0/1156749779 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f36d00a0e20 con 0x7f36d406aa70 2024-03-16T08:16:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:146028888125 2024-03-16T08:16:03.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 msgr2=0x7f36c00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 0x7f36c00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36d4100d50 tx=0x7f36c400b3c0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 msgr2=0x7f36d41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006010 tx=0x7f36d0009f80 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 shutdown_connections 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f36c00a29a0 0x7f36c00a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36d4107650 0x7f36d4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d406aa70 0x7f36d41012d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 --2- 172.21.15.49:0/1156749779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d406b420 0x7f36d4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 >> 172.21.15.49:0/1156749779 conn(0x7f36d4077220 msgr2=0x7f36d4108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 shutdown_connections 2024-03-16T08:16:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.379+0000 7f36db624700 1 -- 172.21.15.49:0/1156749779 wait complete. 2024-03-16T08:16:03.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:03 smithi154 ceph-mon[36471]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 -- 172.21.15.49:0/59394340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 msgr2=0x7ffbd806b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/59394340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd806b010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80099c0 tx=0x7ffbc8009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 -- 172.21.15.49:0/59394340 shutdown_connections 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/59394340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8107680 0x7ffbd810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/59394340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd806b010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/59394340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd806b5e0 0x7ffbd8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 -- 172.21.15.49:0/59394340 >> 172.21.15.49:0/59394340 conn(0x7ffbd8077220 msgr2=0x7ffbd8077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 -- 172.21.15.49:0/59394340 shutdown_connections 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.843+0000 7ffbdcc60700 1 -- 172.21.15.49:0/59394340 wait complete. 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 Processor -- start 2024-03-16T08:16:03.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 -- start start 2024-03-16T08:16:03.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd806b5e0 0x7ffbd8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8107680 0x7ffbd8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8112610 con 0x7ffbd806ac30 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8112490 con 0x7ffbd8107680 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbdcc60700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8112790 con 0x7ffbd806b5e0 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39382/0 (socket says 172.21.15.49:39382) 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd806b5e0 0x7ffbd8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 -- 172.21.15.49:0/1790543607 learned_addr learned my addr 172.21.15.49:0/1790543607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd6d9d700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8107680 0x7ffbd8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 -- 172.21.15.49:0/1790543607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd806b5e0 msgr2=0x7ffbd8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd806b5e0 0x7ffbd8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.844+0000 7ffbd659c700 1 -- 172.21.15.49:0/1790543607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8107680 msgr2=0x7ffbd8101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbd659c700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8107680 0x7ffbd8101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbd659c700 1 -- 172.21.15.49:0/1790543607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd81a7050 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbd659c700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80059d0 tx=0x7ffbc80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8027070 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8003e40 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd81a72e0 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8022d00 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.845+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81a7790 con 0x7ffbd806ac30 2024-03-16T08:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.846+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd810a830 con 0x7ffbd806ac30 2024-03-16T08:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.847+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ffbc802a070 con 0x7ffbd806ac30 2024-03-16T08:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.848+0000 7ffbc77fe700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 0x7ffbc00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.848+0000 7ffbd5d9b700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 0x7ffbc00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.848+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffbc80d0000 con 0x7ffbd806ac30 2024-03-16T08:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.848+0000 7ffbd5d9b700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 0x7ffbc00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8100e20 tx=0x7ffbcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:03.851+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffbc8031020 con 0x7ffbd806ac30 2024-03-16T08:16:04.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.064+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ffbd8102830 con 0x7ffbd806ac30 2024-03-16T08:16:04.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.064+0000 7ffbc77fe700 1 -- 172.21.15.49:0/1790543607 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffbc8013070 con 0x7ffbd806ac30 2024-03-16T08:16:04.066 INFO:teuthology.orchestra.run.smithi049.stdout:347892351001 2024-03-16T08:16:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 msgr2=0x7ffbc00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 0x7ffbc00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8100e20 tx=0x7ffbcc00b410 comp rx=0 tx=0).stop 2024-03-16T08:16:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 msgr2=0x7ffbd8101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80059d0 tx=0x7ffbc80038b0 comp rx=0 tx=0).stop 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 shutdown_connections 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ffbc00a29a0 0x7ffbc00a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8107680 0x7ffbd8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbd806ac30 0x7ffbd8101300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 --2- 172.21.15.49:0/1790543607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd806b5e0 0x7ffbd8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 >> 172.21.15.49:0/1790543607 conn(0x7ffbd8077220 msgr2=0x7ffbd80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 shutdown_connections 2024-03-16T08:16:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:04.066+0000 7ffbdcc60700 1 -- 172.21.15.49:0/1790543607 wait complete. 2024-03-16T08:16:04.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738447 for osd.0 2024-03-16T08:16:04.255 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1156749779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-16T08:16:04.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1790543607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T08:16:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1156749779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-16T08:16:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1790543607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T08:16:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1156749779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-16T08:16:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1790543607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T08:16:05.107 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724599 for osd.5 2024-03-16T08:16:05.107 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:05.497 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:05.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888117 got 146028888125 for osd.4 2024-03-16T08:16:05.507 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:05.673 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-16T08:16:05.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:05 smithi049 ceph-mon[31942]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T08:16:05.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:05.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T08:16:05.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-16T08:16:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:05 smithi154 ceph-mon[36471]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T08:16:05.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:05.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-16T08:16:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:05 smithi129 ceph-mon[36561]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T08:16:06.597 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:06 smithi049 ceph-mon[31942]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T08:16:06.598 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1827430435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T08:16:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:06 smithi154 ceph-mon[36471]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T08:16:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1827430435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T08:16:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:06 smithi129 ceph-mon[36561]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T08:16:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1827430435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T08:16:07.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:07 smithi049 ceph-mon[31942]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:07.591 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:07 smithi049 ceph-mon[31942]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T08:16:07.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892351001 for osd.11 2024-03-16T08:16:07.626 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:07 smithi154 ceph-mon[36471]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:07.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:07 smithi154 ceph-mon[36471]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- 172.21.15.49:0/1382964524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac101e40 msgr2=0x7fbaac105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/1382964524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac101e40 0x7fbaac105e00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fba980099c0 tx=0x7fba98009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- 172.21.15.49:0/1382964524 shutdown_connections 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/1382964524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac1064e0 0x7fbaac10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/1382964524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101490 0x7fbaac101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/1382964524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac101e40 0x7fbaac105e00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- 172.21.15.49:0/1382964524 >> 172.21.15.49:0/1382964524 conn(0x7fbaac077220 msgr2=0x7fbaac077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- 172.21.15.49:0/1382964524 shutdown_connections 2024-03-16T08:16:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- 172.21.15.49:0/1382964524 wait complete. 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 Processor -- start 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.821+0000 7fbab2c7e700 1 -- start start 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac101490 0x7fbaac102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 0x7fbaac1033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 0x7fbaac103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1114a0 con 0x7fbaac1064e0 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111320 con 0x7fbaac101490 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab2c7e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111620 con 0x7fbaac101e40 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab121b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 0x7fbaac103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab121b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 0x7fbaac103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39400/0 (socket says 172.21.15.49:39400) 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab0a1a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac101490 0x7fbaac102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 0x7fbaac1033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbab121b700 1 -- 172.21.15.49:0/2641641377 learned_addr learned my addr 172.21.15.49:0/2641641377 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 -- 172.21.15.49:0/2641641377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac101490 msgr2=0x7fbaac102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac101490 0x7fbaac102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 -- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 msgr2=0x7fbaac103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 0x7fbaac103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 -- 172.21.15.49:0/2641641377 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac1a6f50 con 0x7fbaac101e40 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.822+0000 7fbaabfff700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 0x7fbaac1033e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba980098e0 tx=0x7fba9800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.823+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98027070 con 0x7fbaac101e40 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.823+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba98003680 con 0x7fbaac101e40 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.823+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac1a71e0 con 0x7fbaac101e40 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.823+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980173f0 con 0x7fbaac101e40 2024-03-16T08:16:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.823+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a7630 con 0x7fbaac101e40 2024-03-16T08:16:07.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.824+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac068f90 con 0x7fbaac101e40 2024-03-16T08:16:07.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.825+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fba9802a070 con 0x7fbaac101e40 2024-03-16T08:16:07.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.826+0000 7fbaa9ffb700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 0x7fba9c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.826+0000 7fbab0a1a700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 0x7fba9c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.826+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fba980cf790 con 0x7fbaac101e40 2024-03-16T08:16:07.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.826+0000 7fbab0a1a700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 0x7fba9c0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009c60 tx=0x7fbaa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:07.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:07.830+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba9805e740 con 0x7fbaac101e40 2024-03-16T08:16:07.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:07 smithi129 ceph-mon[36561]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:07.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:07 smithi129 ceph-mon[36561]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T08:16:07.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 -- 172.21.15.49:0/445323886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc1035e0 msgr2=0x7fa6fc1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 --2- 172.21.15.49:0/445323886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc1039c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009990 tx=0x7fa6e4009ca0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 -- 172.21.15.49:0/445323886 shutdown_connections 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 --2- 172.21.15.49:0/445323886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 --2- 172.21.15.49:0/445323886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc1039c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 --2- 172.21.15.49:0/445323886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc103f00 0x7fa6fc10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 -- 172.21.15.49:0/445323886 >> 172.21.15.49:0/445323886 conn(0x7fa6fc078300 msgr2=0x7fa6fc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 -- 172.21.15.49:0/445323886 shutdown_connections 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 -- 172.21.15.49:0/445323886 wait complete. 2024-03-16T08:16:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.123+0000 7fa702119700 1 Processor -- start 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 -- start start 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1126c0 con 0x7fa6fc103f00 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112540 con 0x7fa6fc10c9c0 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa702119700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112840 con 0x7fa6fc1035e0 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39390/0 (socket says 172.21.15.49:39390) 2024-03-16T08:16:07.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 -- 172.21.15.49:0/1827430435 learned_addr learned my addr 172.21.15.49:0/1827430435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:07.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6fbfff700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6fb7fe700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.936 INFO:teuthology.orchestra.run.smithi049.stdout:313532612638 2024-03-16T08:16:07.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 -- 172.21.15.49:0/1827430435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc1035e0 msgr2=0x7fa6fc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 -- 172.21.15.49:0/1827430435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 msgr2=0x7fa6fc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6faffd700 1 -- 172.21.15.49:0/1827430435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1990a0 con 0x7fa6fc103f00 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.124+0000 7fa6fbfff700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa6faffd700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e9f0 tx=0x7fa6ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004a30 con 0x7fa6fc103f00 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc199390 con 0x7fa6fc103f00 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec00cca0 con 0x7fa6fc103f00 2024-03-16T08:16:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0056c0 con 0x7fa6fc103f00 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.125+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1a5650 con 0x7fa6fc103f00 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.126+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc068f90 con 0x7fa6fc103f00 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.127+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fa6ec00c7c0 con 0x7fa6fc103f00 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.128+0000 7fa6f8ff9700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 0x7fa6e80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.128+0000 7fa6fb7fe700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 0x7fa6e80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.128+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fa6ec0cefc0 con 0x7fa6fc103f00 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.128+0000 7fa6fb7fe700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 0x7fa6e80a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b2e0 tx=0x7fa6e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:07.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.131+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6ec053740 con 0x7fa6fc103f00 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.341+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa6fc199e70 con 0x7fa6fc103f00 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.342+0000 7fa6f8ff9700 1 -- 172.21.15.49:0/1827430435 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa6fc199e70 con 0x7fa6fc103f00 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 msgr2=0x7fa6e80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 0x7fa6e80a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b2e0 tx=0x7fa6e401a040 comp rx=0 tx=0).stop 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 msgr2=0x7fa6fc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e9f0 tx=0x7fa6ec00ed00 comp rx=0 tx=0).stop 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 shutdown_connections 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa6e80a2920 0x7fa6e80a4de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6fc10c9c0 0x7fa6fc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc1035e0 0x7fa6fc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 --2- 172.21.15.49:0/1827430435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc103f00 0x7fa6fc19ecc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:07.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 >> 172.21.15.49:0/1827430435 conn(0x7fa6fc078300 msgr2=0x7fa6fc0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:07.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 shutdown_connections 2024-03-16T08:16:07.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:06.344+0000 7fa702119700 1 -- 172.21.15.49:0/1827430435 wait complete. 2024-03-16T08:16:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.040+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbaac02dd30 con 0x7fbaac101e40 2024-03-16T08:16:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.041+0000 7fbaa9ffb700 1 -- 172.21.15.49:0/2641641377 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fba98013070 con 0x7fbaac101e40 2024-03-16T08:16:08.042 INFO:teuthology.orchestra.run.smithi049.stdout:201863462963 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 msgr2=0x7fba9c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 0x7fba9c0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009c60 tx=0x7fbaa0009260 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 msgr2=0x7fbaac1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 0x7fbaac1033e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba980098e0 tx=0x7fba9800fa00 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 shutdown_connections 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fba9c0a2920 0x7fba9c0a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbaac1064e0 0x7fbaac103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaac101490 0x7fbaac102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 --2- 172.21.15.49:0/2641641377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaac101e40 0x7fbaac1033e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 >> 172.21.15.49:0/2641641377 conn(0x7fbaac077220 msgr2=0x7fbaac06ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:08.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 shutdown_connections 2024-03-16T08:16:08.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:08.043+0000 7fbab2c7e700 1 -- 172.21.15.49:0/2641641377 wait complete. 2024-03-16T08:16:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:08 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2641641377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T08:16:08.583 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612638 for osd.10 2024-03-16T08:16:08.583 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:08 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2641641377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T08:16:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:08 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2641641377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T08:16:09.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:09 smithi049 ceph-mon[31942]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:16:09.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:09 smithi154 ceph-mon[36471]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:16:09.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 -- 172.21.15.49:0/1316957215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480107660 msgr2=0x7f048010f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 --2- 172.21.15.49:0/1316957215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480107660 0x7f048010f1e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f047800b0d0 tx=0x7f047800b3e0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 -- 172.21.15.49:0/1316957215 shutdown_connections 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 --2- 172.21.15.49:0/1316957215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480107660 0x7f048010f1e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 --2- 172.21.15.49:0/1316957215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04801025b0 0x7f0480102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 --2- 172.21.15.49:0/1316957215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 0x7f0480106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 -- 172.21.15.49:0/1316957215 >> 172.21.15.49:0/1316957215 conn(0x7f04800793f0 msgr2=0x7f0480079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 -- 172.21.15.49:0/1316957215 shutdown_connections 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 -- 172.21.15.49:0/1316957215 wait complete. 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.836+0000 7f0488430700 1 Processor -- start 2024-03-16T08:16:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 -- start start 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 0x7f04801a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0480107660 0x7f048019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0480112600 con 0x7f04801025b0 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0480112480 con 0x7f0480107660 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f0488430700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0480112780 con 0x7f0480102f60 2024-03-16T08:16:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39428/0 (socket says 172.21.15.49:39428) 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 -- 172.21.15.49:0/2627482135 learned_addr learned my addr 172.21.15.49:0/2627482135 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04869cd700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0480107660 0x7f048019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04859cb700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 0x7f04801a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 -- 172.21.15.49:0/2627482135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 msgr2=0x7f04801a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 0x7f04801a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 -- 172.21.15.49:0/2627482135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0480107660 msgr2=0x7f048019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0480107660 0x7f048019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 -- 172.21.15.49:0/2627482135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048019b1d0 con 0x7f04801025b0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.837+0000 7f04861cc700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9a0 tx=0x7f047c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.838+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ca10 con 0x7f04801025b0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.838+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048019b4c0 con 0x7f04801025b0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.838+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c00cbb0 con 0x7f04801025b0 2024-03-16T08:16:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.838+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c018680 con 0x7f04801025b0 2024-03-16T08:16:09.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.838+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801a7780 con 0x7f04801025b0 2024-03-16T08:16:09.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.839+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480068f90 con 0x7f04801025b0 2024-03-16T08:16:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.840+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f047c010430 con 0x7f04801025b0 2024-03-16T08:16:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.840+0000 7f04777fe700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 0x7f046c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.840+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f047c0cf290 con 0x7f04801025b0 2024-03-16T08:16:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.840+0000 7f04859cb700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 0x7f046c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.841+0000 7f04859cb700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 0x7f046c0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0480108d90 tx=0x7f0470009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:09.844+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f047c09c200 con 0x7f04801025b0 2024-03-16T08:16:09.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:09 smithi129 ceph-mon[36561]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:16:10.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.064+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f04801a79b0 con 0x7f04801025b0 2024-03-16T08:16:10.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.064+0000 7f04777fe700 1 -- 172.21.15.49:0/2627482135 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f047c09bb00 con 0x7f04801025b0 2024-03-16T08:16:10.066 INFO:teuthology.orchestra.run.smithi049.stdout:111669149761 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.066+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 msgr2=0x7f046c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.066+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 0x7f046c0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0480108d90 tx=0x7f0470009260 comp rx=0 tx=0).stop 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.066+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 msgr2=0x7f04801a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.066+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9a0 tx=0x7f047c00ecb0 comp rx=0 tx=0).stop 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 shutdown_connections 2024-03-16T08:16:10.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f046c0a2920 0x7f046c0a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0480107660 0x7f048019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04801025b0 0x7f04801a08b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 --2- 172.21.15.49:0/2627482135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0480102f60 0x7f04801a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 >> 172.21.15.49:0/2627482135 conn(0x7f04800793f0 msgr2=0x7f0480100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 shutdown_connections 2024-03-16T08:16:10.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:10.067+0000 7f0488430700 1 -- 172.21.15.49:0/2627482135 wait complete. 2024-03-16T08:16:10.291 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462963 for osd.6 2024-03-16T08:16:10.291 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:10.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:10 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2627482135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T08:16:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:10 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2627482135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T08:16:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:10 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2627482135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T08:16:11.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:11 smithi154 ceph-mon[36471]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:11.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:11 smithi049 ceph-mon[31942]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:11.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:11 smithi129 ceph-mon[36561]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 -- 172.21.15.49:0/2509703836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5806abd0 msgr2=0x7faf5806afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2509703836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5806abd0 0x7faf5806afb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf480099c0 tx=0x7faf48009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 -- 172.21.15.49:0/2509703836 shutdown_connections 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2509703836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf58107640 0x7faf5810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2509703836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5806abd0 0x7faf5806afb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2509703836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806b580 0x7faf58106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 -- 172.21.15.49:0/2509703836 >> 172.21.15.49:0/2509703836 conn(0x7faf58078140 msgr2=0x7faf58078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:12.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 -- 172.21.15.49:0/2509703836 shutdown_connections 2024-03-16T08:16:12.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.309+0000 7faf5d68f700 1 -- 172.21.15.49:0/2509703836 wait complete. 2024-03-16T08:16:12.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 Processor -- start 2024-03-16T08:16:12.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 -- start start 2024-03-16T08:16:12.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 0x7faf581017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf581125d0 con 0x7faf58107640 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf58112450 con 0x7faf5806abd0 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf5d68f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf58112750 con 0x7faf5806b580 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf56ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39450/0 (socket says 172.21.15.49:39450) 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf577fe700 1 -- 172.21.15.49:0/2321985811 learned_addr learned my addr 172.21.15.49:0/2321985811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf56ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51488/0 (socket says 172.21.15.49:51488) 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.310+0000 7faf567fc700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 0x7faf581017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 -- 172.21.15.49:0/2321985811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 msgr2=0x7faf581017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:12.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 0x7faf581017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 -- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 msgr2=0x7faf58101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 -- 172.21.15.49:0/2321985811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf581ab410 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf577fe700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf567fc700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 0x7faf581017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf56ffd700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faf4800b580 tx=0x7faf48003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48003680 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf480055a0 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf581ab640 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800fc90 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.311+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf581abaf0 con 0x7faf5806abd0 2024-03-16T08:16:12.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.312+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5810a7f0 con 0x7faf5806abd0 2024-03-16T08:16:12.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.314+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7faf4800f460 con 0x7faf5806abd0 2024-03-16T08:16:12.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.314+0000 7faf3ffff700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 0x7faf400a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:12.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.314+0000 7faf567fc700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 0x7faf400a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:12.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.314+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7faf480cfb20 con 0x7faf5806abd0 2024-03-16T08:16:12.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.315+0000 7faf567fc700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 0x7faf400a4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf58102a50 tx=0x7faf4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:12.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.318+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faf48013070 con 0x7faf5806abd0 2024-03-16T08:16:12.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.526+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faf5806a220 con 0x7faf5806abd0 2024-03-16T08:16:12.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.526+0000 7faf3ffff700 1 -- 172.21.15.49:0/2321985811 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faf5806a220 con 0x7faf5806abd0 2024-03-16T08:16:12.528 INFO:teuthology.orchestra.run.smithi049.stdout:292057776165 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 msgr2=0x7faf400a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 0x7faf400a4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf58102a50 tx=0x7faf4c009260 comp rx=0 tx=0).stop 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 msgr2=0x7faf58101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faf4800b580 tx=0x7faf48003a90 comp rx=0 tx=0).stop 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 shutdown_connections 2024-03-16T08:16:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7faf400a2ac0 0x7faf400a4f80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf58107640 0x7faf58101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf5806abd0 0x7faf58101290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 --2- 172.21.15.49:0/2321985811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5806b580 0x7faf581017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 >> 172.21.15.49:0/2321985811 conn(0x7faf58078140 msgr2=0x7faf580ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 shutdown_connections 2024-03-16T08:16:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:12.529+0000 7faf5d68f700 1 -- 172.21.15.49:0/2321985811 wait complete. 2024-03-16T08:16:12.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:12 smithi154 ceph-mon[36471]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:12.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:12 smithi154 ceph-mon[36471]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-16T08:16:12.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:12 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:16:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:12 smithi049 ceph-mon[31942]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:12 smithi049 ceph-mon[31942]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-16T08:16:12.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:12 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:16:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:12 smithi129 ceph-mon[36561]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:12 smithi129 ceph-mon[36561]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-16T08:16:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:12 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:16:13.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149754 got 111669149761 for osd.3 2024-03-16T08:16:13.178 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:13.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776165 for osd.9 2024-03-16T08:16:13.601 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:13.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:13 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2321985811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T08:16:13.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:13 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2321985811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T08:16:13.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:13 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2321985811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T08:16:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- 172.21.15.49:0/287972737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101cc0 msgr2=0x7fab8c105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 --2- 172.21.15.49:0/287972737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101cc0 0x7fab8c105d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab780099c0 tx=0x7fab78009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- 172.21.15.49:0/287972737 shutdown_connections 2024-03-16T08:16:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 --2- 172.21.15.49:0/287972737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c1063f0 0x7fab8c10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 --2- 172.21.15.49:0/287972737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c101310 0x7fab8c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 --2- 172.21.15.49:0/287972737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101cc0 0x7fab8c105d10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- 172.21.15.49:0/287972737 >> 172.21.15.49:0/287972737 conn(0x7fab8c0fcba0 msgr2=0x7fab8c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- 172.21.15.49:0/287972737 shutdown_connections 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- 172.21.15.49:0/287972737 wait complete. 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 Processor -- start 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.331+0000 7fab9199d700 1 -- start start 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c101cc0 0x7fab8c0fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c1063f0 0x7fab8c0fe010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111400 con 0x7fab8c101310 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111280 con 0x7fab8c1063f0 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9199d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111580 con 0x7fab8c101cc0 2024-03-16T08:16:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56202/0 (socket says 172.21.15.49:56202) 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 -- 172.21.15.49:0/1983894075 learned_addr learned my addr 172.21.15.49:0/1983894075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab8bfff700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c101cc0 0x7fab8c0fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9119c700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c1063f0 0x7fab8c0fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 -- 172.21.15.49:0/1983894075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c101cc0 msgr2=0x7fab8c0fdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c101cc0 0x7fab8c0fdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 -- 172.21.15.49:0/1983894075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c1063f0 msgr2=0x7fab8c0fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c1063f0 0x7fab8c0fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 -- 172.21.15.49:0/1983894075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c1a6ef0 con 0x7fab8c101310 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab9099b700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab8000e9a0 tx=0x7fab8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.332+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8000ca10 con 0x7fab8c101310 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.333+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1a71e0 con 0x7fab8c101310 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.333+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8000cbb0 con 0x7fab8c101310 2024-03-16T08:16:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.333+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80018680 con 0x7fab8c101310 2024-03-16T08:16:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.333+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a7740 con 0x7fab8c101310 2024-03-16T08:16:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.334+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c068f90 con 0x7fab8c101310 2024-03-16T08:16:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.335+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fab80010430 con 0x7fab8c101310 2024-03-16T08:16:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.335+0000 7fab89ffb700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 0x7fab7c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.336+0000 7fab8bfff700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 0x7fab7c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.336+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fab800cf290 con 0x7fab8c101310 2024-03-16T08:16:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.336+0000 7fab8bfff700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 0x7fab7c0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab78000c00 tx=0x7fab78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.339+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fab8009c200 con 0x7fab8c101310 2024-03-16T08:16:14.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.559+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab8c02db70 con 0x7fab8c101310 2024-03-16T08:16:14.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.559+0000 7fab89ffb700 1 -- 172.21.15.49:0/1983894075 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab8009bb00 con 0x7fab8c101310 2024-03-16T08:16:14.561 INFO:teuthology.orchestra.run.smithi049.stdout:77309411401 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 msgr2=0x7fab7c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 0x7fab7c0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab78000c00 tx=0x7fab78005fb0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 msgr2=0x7fab8c0fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab8000e9a0 tx=0x7fab8000ecb0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 shutdown_connections 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab7c0a2920 0x7fab7c0a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8c1063f0 0x7fab8c0fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c101310 0x7fab8c0fd590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 --2- 172.21.15.49:0/1983894075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c101cc0 0x7fab8c0fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 >> 172.21.15.49:0/1983894075 conn(0x7fab8c0fcba0 msgr2=0x7fab8c06aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:14.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 shutdown_connections 2024-03-16T08:16:14.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.562+0000 7fab9199d700 1 -- 172.21.15.49:0/1983894075 wait complete. 2024-03-16T08:16:14.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 -- 172.21.15.49:0/116994454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4006a4c0 msgr2=0x7f2b4006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 --2- 172.21.15.49:0/116994454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4006a4c0 0x7f2b4006a920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009990 tx=0x7f2b34009ca0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 -- 172.21.15.49:0/116994454 shutdown_connections 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 --2- 172.21.15.49:0/116994454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b40104cd0 0x7f2b40105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 --2- 172.21.15.49:0/116994454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 0x7f2b40069f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.606+0000 7f2b44e68700 1 --2- 172.21.15.49:0/116994454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4006a4c0 0x7f2b4006a920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.607+0000 7f2b44e68700 1 -- 172.21.15.49:0/116994454 >> 172.21.15.49:0/116994454 conn(0x7f2b40078100 msgr2=0x7f2b40078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.607+0000 7f2b44e68700 1 -- 172.21.15.49:0/116994454 shutdown_connections 2024-03-16T08:16:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.607+0000 7f2b44e68700 1 -- 172.21.15.49:0/116994454 wait complete. 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.607+0000 7f2b44e68700 1 Processor -- start 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 -- start start 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b4006a4c0 0x7f2b4019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 0x7f2b401a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40118b70 con 0x7f2b40104cd0 2024-03-16T08:16:14.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401189f0 con 0x7f2b4006a4c0 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b44e68700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40118cf0 con 0x7f2b401071c0 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b3dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b3ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 0x7f2b401a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b3dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56226/0 (socket says 172.21.15.49:56226) 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b3dd9b700 1 -- 172.21.15.49:0/2763452416 learned_addr learned my addr 172.21.15.49:0/2763452416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.608+0000 7f2b3e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b4006a4c0 0x7f2b4019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 -- 172.21.15.49:0/2763452416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 msgr2=0x7f2b401a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 0x7f2b401a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 -- 172.21.15.49:0/2763452416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b4006a4c0 msgr2=0x7f2b4019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b4006a4c0 0x7f2b4019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 -- 172.21.15.49:0/2763452416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b401a40f0 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b3dd9b700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b340038b0 tx=0x7f2b34009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34027070 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34004050 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b401a4380 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34022c00 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.609+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401abc60 con 0x7f2b40104cd0 2024-03-16T08:16:14.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.610+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b40068f90 con 0x7f2b40104cd0 2024-03-16T08:16:14.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.612+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2b3402a070 con 0x7f2b40104cd0 2024-03-16T08:16:14.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.613+0000 7f2b2f7fe700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 0x7f2b280a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:14.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.613+0000 7f2b3e59c700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 0x7f2b280a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:14.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.613+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f2b340d01d0 con 0x7f2b40104cd0 2024-03-16T08:16:14.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.613+0000 7f2b3e59c700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 0x7f2b280a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b30007b60 tx=0x7f2b300073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:14.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.616+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b34031020 con 0x7f2b40104cd0 2024-03-16T08:16:14.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:14 smithi049 ceph-mon[31942]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:14.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.828+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2b401abe90 con 0x7f2b40104cd0 2024-03-16T08:16:14.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.828+0000 7f2b2f7fe700 1 -- 172.21.15.49:0/2763452416 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2b34013070 con 0x7f2b40104cd0 2024-03-16T08:16:14.830 INFO:teuthology.orchestra.run.smithi049.stdout:223338299439 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 msgr2=0x7f2b280a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 0x7f2b280a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b30007b60 tx=0x7f2b300073d0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 msgr2=0x7f2b4019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b340038b0 tx=0x7f2b34009f50 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 shutdown_connections 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2b280a29a0 0x7f2b280a4e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b401071c0 0x7f2b401a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b4006a4c0 0x7f2b4019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 --2- 172.21.15.49:0/2763452416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b40104cd0 0x7f2b4019f620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.830+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 >> 172.21.15.49:0/2763452416 conn(0x7f2b40078100 msgr2=0x7f2b40103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.831+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 shutdown_connections 2024-03-16T08:16:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:14.831+0000 7f2b44e68700 1 -- 172.21.15.49:0/2763452416 wait complete. 2024-03-16T08:16:14.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:14 smithi129 ceph-mon[36561]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:14.913 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:14 smithi154 ceph-mon[36471]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:15.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411401 for osd.2 2024-03-16T08:16:15.205 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:15.761 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:15 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1983894075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T08:16:15.762 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:15 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2763452416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T08:16:15.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:15 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1983894075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T08:16:15.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:15 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2763452416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T08:16:15.876 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299439 for osd.7 2024-03-16T08:16:15.877 DEBUG:teuthology.parallel:result is None 2024-03-16T08:16:15.877 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-16T08:16:15.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T08:16:15.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:15 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1983894075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T08:16:15.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:15 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2763452416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T08:16:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:16.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:16 smithi049 ceph-mon[31942]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:16.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:16 smithi129 ceph-mon[36561]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:16.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:16 smithi154 ceph-mon[36471]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-16T08:16:18.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 -- 172.21.15.49:0/4041211729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102320 msgr2=0x7f2cf01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:18.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/4041211729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102320 0x7f2cf01027a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80099c0 tx=0x7f2cd8009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 -- 172.21.15.49:0/4041211729 shutdown_connections 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/4041211729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102ce0 0x7f2cf010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/4041211729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 0x7f2cf01086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/4041211729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102320 0x7f2cf01027a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 -- 172.21.15.49:0/4041211729 >> 172.21.15.49:0/4041211729 conn(0x7f2cf0079400 msgr2=0x7f2cf0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 -- 172.21.15.49:0/4041211729 shutdown_connections 2024-03-16T08:16:18.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.263+0000 7f2cf48bd700 1 -- 172.21.15.49:0/4041211729 wait complete. 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 Processor -- start 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 -- start start 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 0x7f2cf00745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 0x7f2cf0075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf01125a0 con 0x7f2cf0102ce0 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0112420 con 0x7f2cf0102320 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cf48bd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0112720 con 0x7f2cf0108300 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cedd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cee59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 0x7f2cf00745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cedd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56232/0 (socket says 172.21.15.49:56232) 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2cedd9b700 1 -- 172.21.15.49:0/462380015 learned_addr learned my addr 172.21.15.49:0/462380015 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:18.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.264+0000 7f2ceed9d700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 0x7f2cf0075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 -- 172.21.15.49:0/462380015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 msgr2=0x7f2cf0075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 0x7f2cf0075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 -- 172.21.15.49:0/462380015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 msgr2=0x7f2cf00745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 0x7f2cf00745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 -- 172.21.15.49:0/462380015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a6f60 con 0x7f2cf0102ce0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cee59c700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 0x7f2cf00745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cedd9b700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8009e60 con 0x7f2cf0102ce0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a71f0 con 0x7f2cf0102ce0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8003680 con 0x7f2cf0102ce0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80173f0 con 0x7f2cf0102ce0 2024-03-16T08:16:18.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.265+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a76a0 con 0x7f2cf0102ce0 2024-03-16T08:16:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.266+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf0068f90 con 0x7f2cf0102ce0 2024-03-16T08:16:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.268+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2cd800f680 con 0x7f2cf0102ce0 2024-03-16T08:16:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.268+0000 7f2ce77fe700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 0x7f2cdc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.268+0000 7f2cee59c700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 0x7f2cdc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:18.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.268+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f2cd80cfd10 con 0x7f2cf0102ce0 2024-03-16T08:16:18.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.269+0000 7f2cee59c700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 0x7f2cdc0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8003e30 tx=0x7f2ce800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:18.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.272+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2cd8013070 con 0x7f2cf0102ce0 2024-03-16T08:16:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.495+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cf0062710 con 0x7f2cdc0a2920 2024-03-16T08:16:18.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.503+0000 7f2ce77fe700 1 -- 172.21.15.49:0/462380015 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258936 (secure 0 0 0) 0x7f2cf0062710 con 0x7f2cdc0a2920 2024-03-16T08:16:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 msgr2=0x7f2cdc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 0x7f2cdc0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8003e30 tx=0x7f2ce800b040 comp rx=0 tx=0).stop 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 msgr2=0x7f2cf0074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8003e00 comp rx=0 tx=0).stop 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 shutdown_connections 2024-03-16T08:16:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2cdc0a2920 0x7f2cdc0a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf0108300 0x7f2cf0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cf0102320 0x7f2cf00745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 --2- 172.21.15.49:0/462380015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cf0102ce0 0x7f2cf0074b10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 >> 172.21.15.49:0/462380015 conn(0x7f2cf0079400 msgr2=0x7f2cf0100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 shutdown_connections 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:18.506+0000 7f2cf48bd700 1 -- 172.21.15.49:0/462380015 wait complete. 2024-03-16T08:16:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:16:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:18 smithi049 ceph-mon[31942]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:18 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:18 smithi154 ceph-mon[36471]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:18.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:18.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:18 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:19.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:18 smithi129 ceph-mon[36561]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:19.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:19.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:18 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:19.148 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-03-16T08:16:17.598054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":578,"num_read_kb":408,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356724,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585100,"statfs":{"total":1151940427776,"available":1151575142400,"internally_reserved":0,"allocated":11931648,"data_stored":7470279,"data_compressed":113889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":39,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386233+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:16:05.386233+0000","last_peered":"2024-03-16T08:16:05.386233+0000","last_clean":"2024-03-16T08:16:05.386233+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:16:05.386233+0000","last_undegraded":"2024-03-16T08:16:05.386233+0000","last_fullsized":"2024-03-16T08:16:05.386233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386306+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:16:05.386306+0000","last_peered":"2024-03-16T08:16:05.386306+0000","last_clean":"2024-03-16T08:16:05.386306+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:16:05.386306+0000","last_undegraded":"2024-03-16T08:16:05.386306+0000","last_fullsized":"2024-03-16T08:16:05.386306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385864+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:16:05.385864+0000","last_peered":"2024-03-16T08:16:05.385864+0000","last_clean":"2024-03-16T08:16:05.385864+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:16:05.385864+0000","last_undegraded":"2024-03-16T08:16:05.385864+0000","last_fullsized":"2024-03-16T08:16:05.385864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387926+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:16:05.387926+0000","last_peered":"2024-03-16T08:16:05.387926+0000","last_clean":"2024-03-16T08:16:05.387926+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:16:05.387926+0000","last_undegraded":"2024-03-16T08:16:05.387926+0000","last_fullsized":"2024-03-16T08:16:05.387926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387720+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:16:05.387720+0000","last_peered":"2024-03-16T08:16:05.387720+0000","last_clean":"2024-03-16T08:16:05.387720+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:05.387720+0000","last_undegraded":"2024-03-16T08:16:05.387720+0000","last_fullsized":"2024-03-16T08:16:05.387720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383502+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:16:05.383502+0000","last_peered":"2024-03-16T08:16:05.383502+0000","last_clean":"2024-03-16T08:16:05.383502+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:16:05.383502+0000","last_undegraded":"2024-03-16T08:16:05.383502+0000","last_fullsized":"2024-03-16T08:16:05.383502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380771+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:16:05.380771+0000","last_peered":"2024-03-16T08:16:05.380771+0000","last_clean":"2024-03-16T08:16:05.380771+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:16:05.380771+0000","last_undegraded":"2024-03-16T08:16:05.380771+0000","last_fullsized":"2024-03-16T08:16:05.380771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383735+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:16:05.383735+0000","last_peered":"2024-03-16T08:16:05.383735+0000","last_clean":"2024-03-16T08:16:05.383735+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:16:05.383735+0000","last_undegraded":"2024-03-16T08:16:05.383735+0000","last_fullsized":"2024-03-16T08:16:05.383735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653295+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:16:05.653295+0000","last_peered":"2024-03-16T08:16:05.653295+0000","last_clean":"2024-03-16T08:16:05.653295+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:16:05.653295+0000","last_undegraded":"2024-03-16T08:16:05.653295+0000","last_fullsized":"2024-03-16T08:16:05.653295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383745+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:16:05.383745+0000","last_peered":"2024-03-16T08:16:05.383745+0000","last_clean":"2024-03-16T08:16:05.383745+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:16:05.383745+0000","last_undegraded":"2024-03-16T08:16:05.383745+0000","last_fullsized":"2024-03-16T08:16:05.383745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381136+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:16:05.381136+0000","last_peered":"2024-03-16T08:16:05.381136+0000","last_clean":"2024-03-16T08:16:05.381136+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:16:05.381136+0000","last_undegraded":"2024-03-16T08:16:05.381136+0000","last_fullsized":"2024-03-16T08:16:05.381136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388059+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:16:05.388059+0000","last_peered":"2024-03-16T08:16:05.388059+0000","last_clean":"2024-03-16T08:16:05.388059+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:16:05.388059+0000","last_undegraded":"2024-03-16T08:16:05.388059+0000","last_fullsized":"2024-03-16T08:16:05.388059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389606+0000","last_change":"2024-03-16T08:16:05.389606+0000","last_active":"2024-03-16T08:16:05.389606+0000","last_peered":"2024-03-16T08:16:05.389606+0000","last_clean":"2024-03-16T08:16:05.389606+0000","last_became_active":"2024-03-16T08:16:05.389062+0000","last_became_peered":"2024-03-16T08:16:05.389062+0000","last_unstale":"2024-03-16T08:16:05.389606+0000","last_undegraded":"2024-03-16T08:16:05.389606+0000","last_fullsized":"2024-03-16T08:16:05.389606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:04:08.093332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385284+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:16:05.385284+0000","last_peered":"2024-03-16T08:16:05.385284+0000","last_clean":"2024-03-16T08:16:05.385284+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:16:05.385284+0000","last_undegraded":"2024-03-16T08:16:05.385284+0000","last_fullsized":"2024-03-16T08:16:05.385284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.654998+0000","last_change":"2024-03-16T08:16:05.654998+0000","last_active":"2024-03-16T08:16:05.654998+0000","last_peered":"2024-03-16T08:16:05.654998+0000","last_clean":"2024-03-16T08:16:05.654998+0000","last_became_active":"2024-03-16T08:16:05.654444+0000","last_became_peered":"2024-03-16T08:16:05.654444+0000","last_unstale":"2024-03-16T08:16:05.654998+0000","last_undegraded":"2024-03-16T08:16:05.654998+0000","last_fullsized":"2024-03-16T08:16:05.654998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:04.343402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652919+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:16:05.652919+0000","last_peered":"2024-03-16T08:16:05.652919+0000","last_clean":"2024-03-16T08:16:05.652919+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:16:05.652919+0000","last_undegraded":"2024-03-16T08:16:05.652919+0000","last_fullsized":"2024-03-16T08:16:05.652919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386169+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:16:05.386169+0000","last_peered":"2024-03-16T08:16:05.386169+0000","last_clean":"2024-03-16T08:16:05.386169+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:05.386169+0000","last_undegraded":"2024-03-16T08:16:05.386169+0000","last_fullsized":"2024-03-16T08:16:05.386169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380905+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:16:05.380905+0000","last_peered":"2024-03-16T08:16:05.380905+0000","last_clean":"2024-03-16T08:16:05.380905+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:16:05.380905+0000","last_undegraded":"2024-03-16T08:16:05.380905+0000","last_fullsized":"2024-03-16T08:16:05.380905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653409+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:16:05.653409+0000","last_peered":"2024-03-16T08:16:05.653409+0000","last_clean":"2024-03-16T08:16:05.653409+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:16:05.653409+0000","last_undegraded":"2024-03-16T08:16:05.653409+0000","last_fullsized":"2024-03-16T08:16:05.653409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380775+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:16:05.380775+0000","last_peered":"2024-03-16T08:16:05.380775+0000","last_clean":"2024-03-16T08:16:05.380775+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:16:05.380775+0000","last_undegraded":"2024-03-16T08:16:05.380775+0000","last_fullsized":"2024-03-16T08:16:05.380775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.655479+0000","last_change":"2024-03-16T08:16:05.654774+0000","last_active":"2024-03-16T08:16:05.655479+0000","last_peered":"2024-03-16T08:16:05.655479+0000","last_clean":"2024-03-16T08:16:05.655479+0000","last_became_active":"2024-03-16T08:16:05.654297+0000","last_became_peered":"2024-03-16T08:16:05.654297+0000","last_unstale":"2024-03-16T08:16:05.655479+0000","last_undegraded":"2024-03-16T08:16:05.655479+0000","last_fullsized":"2024-03-16T08:16:05.655479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:50.079111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386863+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:16:05.386863+0000","last_peered":"2024-03-16T08:16:05.386863+0000","last_clean":"2024-03-16T08:16:05.386863+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:16:05.386863+0000","last_undegraded":"2024-03-16T08:16:05.386863+0000","last_fullsized":"2024-03-16T08:16:05.386863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386471+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:16:05.386471+0000","last_peered":"2024-03-16T08:16:05.386471+0000","last_clean":"2024-03-16T08:16:05.386471+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:16:05.386471+0000","last_undegraded":"2024-03-16T08:16:05.386471+0000","last_fullsized":"2024-03-16T08:16:05.386471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388254+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:16:05.388254+0000","last_peered":"2024-03-16T08:16:05.388254+0000","last_clean":"2024-03-16T08:16:05.388254+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:16:05.388254+0000","last_undegraded":"2024-03-16T08:16:05.388254+0000","last_fullsized":"2024-03-16T08:16:05.388254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653092+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:16:05.653092+0000","last_peered":"2024-03-16T08:16:05.653092+0000","last_clean":"2024-03-16T08:16:05.653092+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:16:05.653092+0000","last_undegraded":"2024-03-16T08:16:05.653092+0000","last_fullsized":"2024-03-16T08:16:05.653092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388257+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:16:05.388257+0000","last_peered":"2024-03-16T08:16:05.388257+0000","last_clean":"2024-03-16T08:16:05.388257+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:16:05.388257+0000","last_undegraded":"2024-03-16T08:16:05.388257+0000","last_fullsized":"2024-03-16T08:16:05.388257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389001+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:16:05.389001+0000","last_peered":"2024-03-16T08:16:05.389001+0000","last_clean":"2024-03-16T08:16:05.389001+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:16:05.389001+0000","last_undegraded":"2024-03-16T08:16:05.389001+0000","last_fullsized":"2024-03-16T08:16:05.389001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:14.651289+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:16:14.651289+0000","last_peered":"2024-03-16T08:16:14.651289+0000","last_clean":"2024-03-16T08:16:14.651289+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:16:14.651289+0000","last_undegraded":"2024-03-16T08:16:14.651289+0000","last_fullsized":"2024-03-16T08:16:14.651289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388140+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:16:05.388140+0000","last_peered":"2024-03-16T08:16:05.388140+0000","last_clean":"2024-03-16T08:16:05.388140+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:16:05.388140+0000","last_undegraded":"2024-03-16T08:16:05.388140+0000","last_fullsized":"2024-03-16T08:16:05.388140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383442+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:16:05.383442+0000","last_peered":"2024-03-16T08:16:05.383442+0000","last_clean":"2024-03-16T08:16:05.383442+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:16:05.383442+0000","last_undegraded":"2024-03-16T08:16:05.383442+0000","last_fullsized":"2024-03-16T08:16:05.383442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388869+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:16:05.388869+0000","last_peered":"2024-03-16T08:16:05.388869+0000","last_clean":"2024-03-16T08:16:05.388869+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:16:05.388869+0000","last_undegraded":"2024-03-16T08:16:05.388869+0000","last_fullsized":"2024-03-16T08:16:05.388869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385161+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:16:05.385161+0000","last_peered":"2024-03-16T08:16:05.385161+0000","last_clean":"2024-03-16T08:16:05.385161+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:16:05.385161+0000","last_undegraded":"2024-03-16T08:16:05.385161+0000","last_fullsized":"2024-03-16T08:16:05.385161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383863+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:16:05.383863+0000","last_peered":"2024-03-16T08:16:05.383863+0000","last_clean":"2024-03-16T08:16:05.383863+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:16:05.383863+0000","last_undegraded":"2024-03-16T08:16:05.383863+0000","last_fullsized":"2024-03-16T08:16:05.383863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387954+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:16:05.387954+0000","last_peered":"2024-03-16T08:16:05.387954+0000","last_clean":"2024-03-16T08:16:05.387954+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:16:05.387954+0000","last_undegraded":"2024-03-16T08:16:05.387954+0000","last_fullsized":"2024-03-16T08:16:05.387954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380441+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:16:05.380441+0000","last_peered":"2024-03-16T08:16:05.380441+0000","last_clean":"2024-03-16T08:16:05.380441+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:16:05.380441+0000","last_undegraded":"2024-03-16T08:16:05.380441+0000","last_fullsized":"2024-03-16T08:16:05.380441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:15.651853+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:16:15.651853+0000","last_peered":"2024-03-16T08:16:15.651853+0000","last_clean":"2024-03-16T08:16:15.651853+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:16:15.651853+0000","last_undegraded":"2024-03-16T08:16:15.651853+0000","last_fullsized":"2024-03-16T08:16:15.651853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653236+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:16:05.653236+0000","last_peered":"2024-03-16T08:16:05.653236+0000","last_clean":"2024-03-16T08:16:05.653236+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:16:05.653236+0000","last_undegraded":"2024-03-16T08:16:05.653236+0000","last_fullsized":"2024-03-16T08:16:05.653236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381137+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:16:05.381137+0000","last_peered":"2024-03-16T08:16:05.381137+0000","last_clean":"2024-03-16T08:16:05.381137+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:16:05.381137+0000","last_undegraded":"2024-03-16T08:16:05.381137+0000","last_fullsized":"2024-03-16T08:16:05.381137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386345+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:16:05.386345+0000","last_peered":"2024-03-16T08:16:05.386345+0000","last_clean":"2024-03-16T08:16:05.386345+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:16:05.386345+0000","last_undegraded":"2024-03-16T08:16:05.386345+0000","last_fullsized":"2024-03-16T08:16:05.386345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389097+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:16:05.389097+0000","last_peered":"2024-03-16T08:16:05.389097+0000","last_clean":"2024-03-16T08:16:05.389097+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:16:05.389097+0000","last_undegraded":"2024-03-16T08:16:05.389097+0000","last_fullsized":"2024-03-16T08:16:05.389097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653199+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:16:05.653199+0000","last_peered":"2024-03-16T08:16:05.653199+0000","last_clean":"2024-03-16T08:16:05.653199+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:16:05.653199+0000","last_undegraded":"2024-03-16T08:16:05.653199+0000","last_fullsized":"2024-03-16T08:16:05.653199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":195,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:16.652963+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:16:16.652963+0000","last_peered":"2024-03-16T08:16:16.652963+0000","last_clean":"2024-03-16T08:16:16.652963+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:16:16.652963+0000","last_undegraded":"2024-03-16T08:16:16.652963+0000","last_fullsized":"2024-03-16T08:16:16.652963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":148,"num_read_kb":133,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387977+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:16:05.387977+0000","last_peered":"2024-03-16T08:16:05.387977+0000","last_clean":"2024-03-16T08:16:05.387977+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:16:05.387977+0000","last_undegraded":"2024-03-16T08:16:05.387977+0000","last_fullsized":"2024-03-16T08:16:05.387977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653483+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:16:05.653483+0000","last_peered":"2024-03-16T08:16:05.653483+0000","last_clean":"2024-03-16T08:16:05.653483+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:16:05.653483+0000","last_undegraded":"2024-03-16T08:16:05.653483+0000","last_fullsized":"2024-03-16T08:16:05.653483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385464+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:16:05.385464+0000","last_peered":"2024-03-16T08:16:05.385464+0000","last_clean":"2024-03-16T08:16:05.385464+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:16:05.385464+0000","last_undegraded":"2024-03-16T08:16:05.385464+0000","last_fullsized":"2024-03-16T08:16:05.385464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388595+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:16:05.388595+0000","last_peered":"2024-03-16T08:16:05.388595+0000","last_clean":"2024-03-16T08:16:05.388595+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:16:05.388595+0000","last_undegraded":"2024-03-16T08:16:05.388595+0000","last_fullsized":"2024-03-16T08:16:05.388595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381102+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:16:05.381102+0000","last_peered":"2024-03-16T08:16:05.381102+0000","last_clean":"2024-03-16T08:16:05.381102+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:16:05.381102+0000","last_undegraded":"2024-03-16T08:16:05.381102+0000","last_fullsized":"2024-03-16T08:16:05.381102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:14.668146+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:16:14.668146+0000","last_peered":"2024-03-16T08:16:14.668146+0000","last_clean":"2024-03-16T08:16:14.668146+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:16:14.668146+0000","last_undegraded":"2024-03-16T08:16:14.668146+0000","last_fullsized":"2024-03-16T08:16:14.668146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380746+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:16:05.380746+0000","last_peered":"2024-03-16T08:16:05.380746+0000","last_clean":"2024-03-16T08:16:05.380746+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:16:05.380746+0000","last_undegraded":"2024-03-16T08:16:05.380746+0000","last_fullsized":"2024-03-16T08:16:05.380746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387994+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:16:05.387994+0000","last_peered":"2024-03-16T08:16:05.387994+0000","last_clean":"2024-03-16T08:16:05.387994+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:16:05.387994+0000","last_undegraded":"2024-03-16T08:16:05.387994+0000","last_fullsized":"2024-03-16T08:16:05.387994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388448+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:16:05.388448+0000","last_peered":"2024-03-16T08:16:05.388448+0000","last_clean":"2024-03-16T08:16:05.388448+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:16:05.388448+0000","last_undegraded":"2024-03-16T08:16:05.388448+0000","last_fullsized":"2024-03-16T08:16:05.388448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.394865+0000","last_change":"2024-03-16T08:16:05.394457+0000","last_active":"2024-03-16T08:16:05.394865+0000","last_peered":"2024-03-16T08:16:05.394865+0000","last_clean":"2024-03-16T08:16:05.394865+0000","last_became_active":"2024-03-16T08:16:05.390239+0000","last_became_peered":"2024-03-16T08:16:05.390239+0000","last_unstale":"2024-03-16T08:16:05.394865+0000","last_undegraded":"2024-03-16T08:16:05.394865+0000","last_fullsized":"2024-03-16T08:16:05.394865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:29:29.442138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386139+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:16:05.386139+0000","last_peered":"2024-03-16T08:16:05.386139+0000","last_clean":"2024-03-16T08:16:05.386139+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:16:05.386139+0000","last_undegraded":"2024-03-16T08:16:05.386139+0000","last_fullsized":"2024-03-16T08:16:05.386139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.390744+0000","last_change":"2024-03-16T08:16:05.390744+0000","last_active":"2024-03-16T08:16:05.390744+0000","last_peered":"2024-03-16T08:16:05.390744+0000","last_clean":"2024-03-16T08:16:05.390744+0000","last_became_active":"2024-03-16T08:16:05.390378+0000","last_became_peered":"2024-03-16T08:16:05.390378+0000","last_unstale":"2024-03-16T08:16:05.390744+0000","last_undegraded":"2024-03-16T08:16:05.390744+0000","last_fullsized":"2024-03-16T08:16:05.390744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:13:29.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383634+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:16:05.383634+0000","last_peered":"2024-03-16T08:16:05.383634+0000","last_clean":"2024-03-16T08:16:05.383634+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:16:05.383634+0000","last_undegraded":"2024-03-16T08:16:05.383634+0000","last_fullsized":"2024-03-16T08:16:05.383634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380895+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:16:05.380895+0000","last_peered":"2024-03-16T08:16:05.380895+0000","last_clean":"2024-03-16T08:16:05.380895+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:16:05.380895+0000","last_undegraded":"2024-03-16T08:16:05.380895+0000","last_fullsized":"2024-03-16T08:16:05.380895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389020+0000","last_change":"2024-03-16T08:16:05.389020+0000","last_active":"2024-03-16T08:16:05.389020+0000","last_peered":"2024-03-16T08:16:05.389020+0000","last_clean":"2024-03-16T08:16:05.389020+0000","last_became_active":"2024-03-16T08:16:05.388668+0000","last_became_peered":"2024-03-16T08:16:05.388668+0000","last_unstale":"2024-03-16T08:16:05.389020+0000","last_undegraded":"2024-03-16T08:16:05.389020+0000","last_fullsized":"2024-03-16T08:16:05.389020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:34:04.661388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388171+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:16:05.388171+0000","last_peered":"2024-03-16T08:16:05.388171+0000","last_clean":"2024-03-16T08:16:05.388171+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:16:05.388171+0000","last_undegraded":"2024-03-16T08:16:05.388171+0000","last_fullsized":"2024-03-16T08:16:05.388171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:13.666580+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:16:13.666580+0000","last_peered":"2024-03-16T08:16:13.666580+0000","last_clean":"2024-03-16T08:16:13.666580+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:16:13.666580+0000","last_undegraded":"2024-03-16T08:16:13.666580+0000","last_fullsized":"2024-03-16T08:16:13.666580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387907+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:16:05.387907+0000","last_peered":"2024-03-16T08:16:05.387907+0000","last_clean":"2024-03-16T08:16:05.387907+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:16:05.387907+0000","last_undegraded":"2024-03-16T08:16:05.387907+0000","last_fullsized":"2024-03-16T08:16:05.387907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387927+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:16:05.387927+0000","last_peered":"2024-03-16T08:16:05.387927+0000","last_clean":"2024-03-16T08:16:05.387927+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:16:05.387927+0000","last_undegraded":"2024-03-16T08:16:05.387927+0000","last_fullsized":"2024-03-16T08:16:05.387927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380815+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:16:05.380815+0000","last_peered":"2024-03-16T08:16:05.380815+0000","last_clean":"2024-03-16T08:16:05.380815+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:16:05.380815+0000","last_undegraded":"2024-03-16T08:16:05.380815+0000","last_fullsized":"2024-03-16T08:16:05.380815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385044+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:16:05.385044+0000","last_peered":"2024-03-16T08:16:05.385044+0000","last_clean":"2024-03-16T08:16:05.385044+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:16:05.385044+0000","last_undegraded":"2024-03-16T08:16:05.385044+0000","last_fullsized":"2024-03-16T08:16:05.385044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380918+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:16:05.380918+0000","last_peered":"2024-03-16T08:16:05.380918+0000","last_clean":"2024-03-16T08:16:05.380918+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:16:05.380918+0000","last_undegraded":"2024-03-16T08:16:05.380918+0000","last_fullsized":"2024-03-16T08:16:05.380918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388079+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:16:05.388079+0000","last_peered":"2024-03-16T08:16:05.388079+0000","last_clean":"2024-03-16T08:16:05.388079+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:16:05.388079+0000","last_undegraded":"2024-03-16T08:16:05.388079+0000","last_fullsized":"2024-03-16T08:16:05.388079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384823+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:16:05.384823+0000","last_peered":"2024-03-16T08:16:05.384823+0000","last_clean":"2024-03-16T08:16:05.384823+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:16:05.384823+0000","last_undegraded":"2024-03-16T08:16:05.384823+0000","last_fullsized":"2024-03-16T08:16:05.384823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380801+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:16:05.380801+0000","last_peered":"2024-03-16T08:16:05.380801+0000","last_clean":"2024-03-16T08:16:05.380801+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:16:05.380801+0000","last_undegraded":"2024-03-16T08:16:05.380801+0000","last_fullsized":"2024-03-16T08:16:05.380801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387003+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:16:05.387003+0000","last_peered":"2024-03-16T08:16:05.387003+0000","last_clean":"2024-03-16T08:16:05.387003+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:16:05.387003+0000","last_undegraded":"2024-03-16T08:16:05.387003+0000","last_fullsized":"2024-03-16T08:16:05.387003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388054+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:16:05.388054+0000","last_peered":"2024-03-16T08:16:05.388054+0000","last_clean":"2024-03-16T08:16:05.388054+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:16:05.388054+0000","last_undegraded":"2024-03-16T08:16:05.388054+0000","last_fullsized":"2024-03-16T08:16:05.388054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385782+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:16:05.385782+0000","last_peered":"2024-03-16T08:16:05.385782+0000","last_clean":"2024-03-16T08:16:05.385782+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:16:05.385782+0000","last_undegraded":"2024-03-16T08:16:05.385782+0000","last_fullsized":"2024-03-16T08:16:05.385782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652991+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:16:05.652991+0000","last_peered":"2024-03-16T08:16:05.652991+0000","last_clean":"2024-03-16T08:16:05.652991+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:16:05.652991+0000","last_undegraded":"2024-03-16T08:16:05.652991+0000","last_fullsized":"2024-03-16T08:16:05.652991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387687+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:16:05.387687+0000","last_peered":"2024-03-16T08:16:05.387687+0000","last_clean":"2024-03-16T08:16:05.387687+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:16:05.387687+0000","last_undegraded":"2024-03-16T08:16:05.387687+0000","last_fullsized":"2024-03-16T08:16:05.387687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384150+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:16:05.384150+0000","last_peered":"2024-03-16T08:16:05.384150+0000","last_clean":"2024-03-16T08:16:05.384150+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:16:05.384150+0000","last_undegraded":"2024-03-16T08:16:05.384150+0000","last_fullsized":"2024-03-16T08:16:05.384150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385920+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:16:05.385920+0000","last_peered":"2024-03-16T08:16:05.385920+0000","last_clean":"2024-03-16T08:16:05.385920+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:16:05.385920+0000","last_undegraded":"2024-03-16T08:16:05.385920+0000","last_fullsized":"2024-03-16T08:16:05.385920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653334+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:16:05.653334+0000","last_peered":"2024-03-16T08:16:05.653334+0000","last_clean":"2024-03-16T08:16:05.653334+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:16:05.653334+0000","last_undegraded":"2024-03-16T08:16:05.653334+0000","last_fullsized":"2024-03-16T08:16:05.653334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384061+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:16:05.384061+0000","last_peered":"2024-03-16T08:16:05.384061+0000","last_clean":"2024-03-16T08:16:05.384061+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:16:05.384061+0000","last_undegraded":"2024-03-16T08:16:05.384061+0000","last_fullsized":"2024-03-16T08:16:05.384061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387802+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:16:05.387802+0000","last_peered":"2024-03-16T08:16:05.387802+0000","last_clean":"2024-03-16T08:16:05.387802+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:16:05.387802+0000","last_undegraded":"2024-03-16T08:16:05.387802+0000","last_fullsized":"2024-03-16T08:16:05.387802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380959+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:16:05.380959+0000","last_peered":"2024-03-16T08:16:05.380959+0000","last_clean":"2024-03-16T08:16:05.380959+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:16:05.380959+0000","last_undegraded":"2024-03-16T08:16:05.380959+0000","last_fullsized":"2024-03-16T08:16:05.380959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653465+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:16:05.653465+0000","last_peered":"2024-03-16T08:16:05.653465+0000","last_clean":"2024-03-16T08:16:05.653465+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:16:05.653465+0000","last_undegraded":"2024-03-16T08:16:05.653465+0000","last_fullsized":"2024-03-16T08:16:05.653465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381173+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:16:05.381173+0000","last_peered":"2024-03-16T08:16:05.381173+0000","last_clean":"2024-03-16T08:16:05.381173+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:16:05.381173+0000","last_undegraded":"2024-03-16T08:16:05.381173+0000","last_fullsized":"2024-03-16T08:16:05.381173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388862+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:16:05.388862+0000","last_peered":"2024-03-16T08:16:05.388862+0000","last_clean":"2024-03-16T08:16:05.388862+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:16:05.388862+0000","last_undegraded":"2024-03-16T08:16:05.388862+0000","last_fullsized":"2024-03-16T08:16:05.388862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380993+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:16:05.380993+0000","last_peered":"2024-03-16T08:16:05.380993+0000","last_clean":"2024-03-16T08:16:05.380993+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:16:05.380993+0000","last_undegraded":"2024-03-16T08:16:05.380993+0000","last_fullsized":"2024-03-16T08:16:05.380993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384729+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:16:05.384729+0000","last_peered":"2024-03-16T08:16:05.384729+0000","last_clean":"2024-03-16T08:16:05.384729+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:16:05.384729+0000","last_undegraded":"2024-03-16T08:16:05.384729+0000","last_fullsized":"2024-03-16T08:16:05.384729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381103+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:16:05.381103+0000","last_peered":"2024-03-16T08:16:05.381103+0000","last_clean":"2024-03-16T08:16:05.381103+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:16:05.381103+0000","last_undegraded":"2024-03-16T08:16:05.381103+0000","last_fullsized":"2024-03-16T08:16:05.381103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385973+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:16:05.385973+0000","last_peered":"2024-03-16T08:16:05.385973+0000","last_clean":"2024-03-16T08:16:05.385973+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:16:05.385973+0000","last_undegraded":"2024-03-16T08:16:05.385973+0000","last_fullsized":"2024-03-16T08:16:05.385973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653680+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:16:05.653680+0000","last_peered":"2024-03-16T08:16:05.653680+0000","last_clean":"2024-03-16T08:16:05.653680+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:16:05.653680+0000","last_undegraded":"2024-03-16T08:16:05.653680+0000","last_fullsized":"2024-03-16T08:16:05.653680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653237+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:16:05.653237+0000","last_peered":"2024-03-16T08:16:05.653237+0000","last_clean":"2024-03-16T08:16:05.653237+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:16:05.653237+0000","last_undegraded":"2024-03-16T08:16:05.653237+0000","last_fullsized":"2024-03-16T08:16:05.653237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384764+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:16:05.384764+0000","last_peered":"2024-03-16T08:16:05.384764+0000","last_clean":"2024-03-16T08:16:05.384764+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:16:05.384764+0000","last_undegraded":"2024-03-16T08:16:05.384764+0000","last_fullsized":"2024-03-16T08:16:05.384764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384954+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:16:05.384954+0000","last_peered":"2024-03-16T08:16:05.384954+0000","last_clean":"2024-03-16T08:16:05.384954+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:16:05.384954+0000","last_undegraded":"2024-03-16T08:16:05.384954+0000","last_fullsized":"2024-03-16T08:16:05.384954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386103+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:16:05.386103+0000","last_peered":"2024-03-16T08:16:05.386103+0000","last_clean":"2024-03-16T08:16:05.386103+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:16:05.386103+0000","last_undegraded":"2024-03-16T08:16:05.386103+0000","last_fullsized":"2024-03-16T08:16:05.386103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653614+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:16:05.653614+0000","last_peered":"2024-03-16T08:16:05.653614+0000","last_clean":"2024-03-16T08:16:05.653614+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:16:05.653614+0000","last_undegraded":"2024-03-16T08:16:05.653614+0000","last_fullsized":"2024-03-16T08:16:05.653614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.654782+0000","last_change":"2024-03-16T08:16:05.654782+0000","last_active":"2024-03-16T08:16:05.654782+0000","last_peered":"2024-03-16T08:16:05.654782+0000","last_clean":"2024-03-16T08:16:05.654782+0000","last_became_active":"2024-03-16T08:16:05.654117+0000","last_became_peered":"2024-03-16T08:16:05.654117+0000","last_unstale":"2024-03-16T08:16:05.654782+0000","last_undegraded":"2024-03-16T08:16:05.654782+0000","last_fullsized":"2024-03-16T08:16:05.654782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:17:45.678541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388078+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:16:05.388078+0000","last_peered":"2024-03-16T08:16:05.388078+0000","last_clean":"2024-03-16T08:16:05.388078+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:16:05.388078+0000","last_undegraded":"2024-03-16T08:16:05.388078+0000","last_fullsized":"2024-03-16T08:16:05.388078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384015+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:16:05.384015+0000","last_peered":"2024-03-16T08:16:05.384015+0000","last_clean":"2024-03-16T08:16:05.384015+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:16:05.384015+0000","last_undegraded":"2024-03-16T08:16:05.384015+0000","last_fullsized":"2024-03-16T08:16:05.384015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653043+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:16:05.653043+0000","last_peered":"2024-03-16T08:16:05.653043+0000","last_clean":"2024-03-16T08:16:05.653043+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:16:05.653043+0000","last_undegraded":"2024-03-16T08:16:05.653043+0000","last_fullsized":"2024-03-16T08:16:05.653043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384966+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:16:05.384966+0000","last_peered":"2024-03-16T08:16:05.384966+0000","last_clean":"2024-03-16T08:16:05.384966+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:16:05.384966+0000","last_undegraded":"2024-03-16T08:16:05.384966+0000","last_fullsized":"2024-03-16T08:16:05.384966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386061+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:16:05.386061+0000","last_peered":"2024-03-16T08:16:05.386061+0000","last_clean":"2024-03-16T08:16:05.386061+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:16:05.386061+0000","last_undegraded":"2024-03-16T08:16:05.386061+0000","last_fullsized":"2024-03-16T08:16:05.386061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":442,"num_read_kb":289,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195141,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":459215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1111720,"data_compressed":37963,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":459327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1051279,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":458818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":519657,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":460089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":521398,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":459129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1050963,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:16:19.153 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T08:16:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:19 smithi049 ceph-mon[31942]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:16:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:16:19.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:19 smithi049 ceph-mon[31942]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:19 smithi154 ceph-mon[36471]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:16:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:16:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:19 smithi154 ceph-mon[36471]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:20.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:19 smithi129 ceph-mon[36561]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:20.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:16:20.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:16:20.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:19 smithi129 ceph-mon[36561]: from='mgr.14152 172.21.15.49:0/2426061471' entity='mgr.x' 2024-03-16T08:16:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 -- 172.21.15.49:0/4288964291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67641064e0 msgr2=0x7f676410e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 --2- 172.21.15.49:0/4288964291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67641064e0 0x7f676410e0b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f675c0099c0 tx=0x7f675c009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 -- 172.21.15.49:0/4288964291 shutdown_connections 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 --2- 172.21.15.49:0/4288964291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67641064e0 0x7f676410e0b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 --2- 172.21.15.49:0/4288964291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6764101490 0x7f6764101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 --2- 172.21.15.49:0/4288964291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 0x7f6764105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 -- 172.21.15.49:0/4288964291 >> 172.21.15.49:0/4288964291 conn(0x7f6764077220 msgr2=0x7f6764077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 -- 172.21.15.49:0/4288964291 shutdown_connections 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.960+0000 7f676b26f700 1 -- 172.21.15.49:0/4288964291 wait complete. 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 Processor -- start 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 -- start start 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 0x7f6764199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67641064e0 0x7f67641a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67641114f0 con 0x7f6764101490 2024-03-16T08:16:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6764111370 con 0x7f67641064e0 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676b26f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6764111670 con 0x7f6764101e40 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56252/0 (socket says 172.21.15.49:56252) 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 -- 172.21.15.49:0/2016926688 learned_addr learned my addr 172.21.15.49:0/2016926688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676880a700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 0x7f6764199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676980c700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67641064e0 0x7f67641a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 -- 172.21.15.49:0/2016926688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 msgr2=0x7f6764199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 0x7f6764199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 -- 172.21.15.49:0/2016926688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67641064e0 msgr2=0x7f67641a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67641064e0 0x7f67641a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.961+0000 7f676900b700 1 -- 172.21.15.49:0/2016926688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67641a4e10 con 0x7f6764101490 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f676900b700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f675400ef10 tx=0x7f675400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675400c9c0 con 0x7f6764101490 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6754019070 con 0x7f6764101490 2024-03-16T08:16:20.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67641a4fe0 con 0x7f6764101490 2024-03-16T08:16:20.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754007e60 con 0x7f6764101490 2024-03-16T08:16:20.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.962+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a5490 con 0x7f6764101490 2024-03-16T08:16:20.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.963+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764068f90 con 0x7f6764101490 2024-03-16T08:16:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.964+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6754012050 con 0x7f6764101490 2024-03-16T08:16:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.965+0000 7f67627fc700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 0x7f674c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.965+0000 7f676880a700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 0x7f674c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.965+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f67540ceb20 con 0x7f6764101490 2024-03-16T08:16:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.965+0000 7f676880a700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 0x7f674c0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6764107940 tx=0x7f6758009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:20.969+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6754010040 con 0x7f6764101490 2024-03-16T08:16:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.172+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6764062710 con 0x7f674c0a2920 2024-03-16T08:16:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.179+0000 7f67627fc700 1 -- 172.21.15.49:0/2016926688 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258936 (secure 0 0 0) 0x7f6764062710 con 0x7f674c0a2920 2024-03-16T08:16:21.186 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:21.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 msgr2=0x7f674c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 0x7f674c0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6764107940 tx=0x7f6758009260 comp rx=0 tx=0).stop 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 msgr2=0x7f6764199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f675400ef10 tx=0x7f675400c3a0 comp rx=0 tx=0).stop 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 shutdown_connections 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f674c0a2920 0x7f674c0a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67641064e0 0x7f67641a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:21.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6764101490 0x7f6764199850 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:21.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 --2- 172.21.15.49:0/2016926688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764101e40 0x7f6764199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:21.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 >> 172.21.15.49:0/2016926688 conn(0x7f6764077220 msgr2=0x7f67640ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:21.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 shutdown_connections 2024-03-16T08:16:21.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:21.183+0000 7f676b26f700 1 -- 172.21.15.49:0/2016926688 wait complete. 2024-03-16T08:16:21.190 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:16:21.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:21 smithi129 ceph-mon[36561]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:21.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:21 smithi154 ceph-mon[36471]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:21.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:21 smithi049 ceph-mon[31942]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-16T08:16:21.840 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-03-16T08:16:19.598663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":578,"num_read_kb":408,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356724,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585100,"statfs":{"total":1151940427776,"available":1151575142400,"internally_reserved":0,"allocated":11931648,"data_stored":7470279,"data_compressed":113889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386233+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:16:05.386233+0000","last_peered":"2024-03-16T08:16:05.386233+0000","last_clean":"2024-03-16T08:16:05.386233+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:16:05.386233+0000","last_undegraded":"2024-03-16T08:16:05.386233+0000","last_fullsized":"2024-03-16T08:16:05.386233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386306+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:16:05.386306+0000","last_peered":"2024-03-16T08:16:05.386306+0000","last_clean":"2024-03-16T08:16:05.386306+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:16:05.386306+0000","last_undegraded":"2024-03-16T08:16:05.386306+0000","last_fullsized":"2024-03-16T08:16:05.386306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385864+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:16:05.385864+0000","last_peered":"2024-03-16T08:16:05.385864+0000","last_clean":"2024-03-16T08:16:05.385864+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:16:05.385864+0000","last_undegraded":"2024-03-16T08:16:05.385864+0000","last_fullsized":"2024-03-16T08:16:05.385864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387926+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:16:05.387926+0000","last_peered":"2024-03-16T08:16:05.387926+0000","last_clean":"2024-03-16T08:16:05.387926+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:16:05.387926+0000","last_undegraded":"2024-03-16T08:16:05.387926+0000","last_fullsized":"2024-03-16T08:16:05.387926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387720+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:16:05.387720+0000","last_peered":"2024-03-16T08:16:05.387720+0000","last_clean":"2024-03-16T08:16:05.387720+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:05.387720+0000","last_undegraded":"2024-03-16T08:16:05.387720+0000","last_fullsized":"2024-03-16T08:16:05.387720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383502+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:16:05.383502+0000","last_peered":"2024-03-16T08:16:05.383502+0000","last_clean":"2024-03-16T08:16:05.383502+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:16:05.383502+0000","last_undegraded":"2024-03-16T08:16:05.383502+0000","last_fullsized":"2024-03-16T08:16:05.383502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380771+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:16:05.380771+0000","last_peered":"2024-03-16T08:16:05.380771+0000","last_clean":"2024-03-16T08:16:05.380771+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:16:05.380771+0000","last_undegraded":"2024-03-16T08:16:05.380771+0000","last_fullsized":"2024-03-16T08:16:05.380771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383735+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:16:05.383735+0000","last_peered":"2024-03-16T08:16:05.383735+0000","last_clean":"2024-03-16T08:16:05.383735+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:16:05.383735+0000","last_undegraded":"2024-03-16T08:16:05.383735+0000","last_fullsized":"2024-03-16T08:16:05.383735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653295+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:16:05.653295+0000","last_peered":"2024-03-16T08:16:05.653295+0000","last_clean":"2024-03-16T08:16:05.653295+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:16:05.653295+0000","last_undegraded":"2024-03-16T08:16:05.653295+0000","last_fullsized":"2024-03-16T08:16:05.653295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383745+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:16:05.383745+0000","last_peered":"2024-03-16T08:16:05.383745+0000","last_clean":"2024-03-16T08:16:05.383745+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:16:05.383745+0000","last_undegraded":"2024-03-16T08:16:05.383745+0000","last_fullsized":"2024-03-16T08:16:05.383745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381136+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:16:05.381136+0000","last_peered":"2024-03-16T08:16:05.381136+0000","last_clean":"2024-03-16T08:16:05.381136+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:16:05.381136+0000","last_undegraded":"2024-03-16T08:16:05.381136+0000","last_fullsized":"2024-03-16T08:16:05.381136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388059+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:16:05.388059+0000","last_peered":"2024-03-16T08:16:05.388059+0000","last_clean":"2024-03-16T08:16:05.388059+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:16:05.388059+0000","last_undegraded":"2024-03-16T08:16:05.388059+0000","last_fullsized":"2024-03-16T08:16:05.388059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389606+0000","last_change":"2024-03-16T08:16:05.389606+0000","last_active":"2024-03-16T08:16:05.389606+0000","last_peered":"2024-03-16T08:16:05.389606+0000","last_clean":"2024-03-16T08:16:05.389606+0000","last_became_active":"2024-03-16T08:16:05.389062+0000","last_became_peered":"2024-03-16T08:16:05.389062+0000","last_unstale":"2024-03-16T08:16:05.389606+0000","last_undegraded":"2024-03-16T08:16:05.389606+0000","last_fullsized":"2024-03-16T08:16:05.389606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:04:08.093332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385284+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:16:05.385284+0000","last_peered":"2024-03-16T08:16:05.385284+0000","last_clean":"2024-03-16T08:16:05.385284+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:16:05.385284+0000","last_undegraded":"2024-03-16T08:16:05.385284+0000","last_fullsized":"2024-03-16T08:16:05.385284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.654998+0000","last_change":"2024-03-16T08:16:05.654998+0000","last_active":"2024-03-16T08:16:05.654998+0000","last_peered":"2024-03-16T08:16:05.654998+0000","last_clean":"2024-03-16T08:16:05.654998+0000","last_became_active":"2024-03-16T08:16:05.654444+0000","last_became_peered":"2024-03-16T08:16:05.654444+0000","last_unstale":"2024-03-16T08:16:05.654998+0000","last_undegraded":"2024-03-16T08:16:05.654998+0000","last_fullsized":"2024-03-16T08:16:05.654998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:04.343402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652919+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:16:05.652919+0000","last_peered":"2024-03-16T08:16:05.652919+0000","last_clean":"2024-03-16T08:16:05.652919+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:16:05.652919+0000","last_undegraded":"2024-03-16T08:16:05.652919+0000","last_fullsized":"2024-03-16T08:16:05.652919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386169+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:16:05.386169+0000","last_peered":"2024-03-16T08:16:05.386169+0000","last_clean":"2024-03-16T08:16:05.386169+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:05.386169+0000","last_undegraded":"2024-03-16T08:16:05.386169+0000","last_fullsized":"2024-03-16T08:16:05.386169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380905+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:16:05.380905+0000","last_peered":"2024-03-16T08:16:05.380905+0000","last_clean":"2024-03-16T08:16:05.380905+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:16:05.380905+0000","last_undegraded":"2024-03-16T08:16:05.380905+0000","last_fullsized":"2024-03-16T08:16:05.380905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653409+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:16:05.653409+0000","last_peered":"2024-03-16T08:16:05.653409+0000","last_clean":"2024-03-16T08:16:05.653409+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:16:05.653409+0000","last_undegraded":"2024-03-16T08:16:05.653409+0000","last_fullsized":"2024-03-16T08:16:05.653409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380775+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:16:05.380775+0000","last_peered":"2024-03-16T08:16:05.380775+0000","last_clean":"2024-03-16T08:16:05.380775+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:16:05.380775+0000","last_undegraded":"2024-03-16T08:16:05.380775+0000","last_fullsized":"2024-03-16T08:16:05.380775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.655479+0000","last_change":"2024-03-16T08:16:05.654774+0000","last_active":"2024-03-16T08:16:05.655479+0000","last_peered":"2024-03-16T08:16:05.655479+0000","last_clean":"2024-03-16T08:16:05.655479+0000","last_became_active":"2024-03-16T08:16:05.654297+0000","last_became_peered":"2024-03-16T08:16:05.654297+0000","last_unstale":"2024-03-16T08:16:05.655479+0000","last_undegraded":"2024-03-16T08:16:05.655479+0000","last_fullsized":"2024-03-16T08:16:05.655479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:50.079111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386863+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:16:05.386863+0000","last_peered":"2024-03-16T08:16:05.386863+0000","last_clean":"2024-03-16T08:16:05.386863+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:16:05.386863+0000","last_undegraded":"2024-03-16T08:16:05.386863+0000","last_fullsized":"2024-03-16T08:16:05.386863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386471+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:16:05.386471+0000","last_peered":"2024-03-16T08:16:05.386471+0000","last_clean":"2024-03-16T08:16:05.386471+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:16:05.386471+0000","last_undegraded":"2024-03-16T08:16:05.386471+0000","last_fullsized":"2024-03-16T08:16:05.386471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388254+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:16:05.388254+0000","last_peered":"2024-03-16T08:16:05.388254+0000","last_clean":"2024-03-16T08:16:05.388254+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:16:05.388254+0000","last_undegraded":"2024-03-16T08:16:05.388254+0000","last_fullsized":"2024-03-16T08:16:05.388254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653092+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:16:05.653092+0000","last_peered":"2024-03-16T08:16:05.653092+0000","last_clean":"2024-03-16T08:16:05.653092+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:16:05.653092+0000","last_undegraded":"2024-03-16T08:16:05.653092+0000","last_fullsized":"2024-03-16T08:16:05.653092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388257+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:16:05.388257+0000","last_peered":"2024-03-16T08:16:05.388257+0000","last_clean":"2024-03-16T08:16:05.388257+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:16:05.388257+0000","last_undegraded":"2024-03-16T08:16:05.388257+0000","last_fullsized":"2024-03-16T08:16:05.388257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389001+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:16:05.389001+0000","last_peered":"2024-03-16T08:16:05.389001+0000","last_clean":"2024-03-16T08:16:05.389001+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:16:05.389001+0000","last_undegraded":"2024-03-16T08:16:05.389001+0000","last_fullsized":"2024-03-16T08:16:05.389001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:14.651289+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:16:14.651289+0000","last_peered":"2024-03-16T08:16:14.651289+0000","last_clean":"2024-03-16T08:16:14.651289+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:16:14.651289+0000","last_undegraded":"2024-03-16T08:16:14.651289+0000","last_fullsized":"2024-03-16T08:16:14.651289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388140+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:16:05.388140+0000","last_peered":"2024-03-16T08:16:05.388140+0000","last_clean":"2024-03-16T08:16:05.388140+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:16:05.388140+0000","last_undegraded":"2024-03-16T08:16:05.388140+0000","last_fullsized":"2024-03-16T08:16:05.388140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383442+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:16:05.383442+0000","last_peered":"2024-03-16T08:16:05.383442+0000","last_clean":"2024-03-16T08:16:05.383442+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:16:05.383442+0000","last_undegraded":"2024-03-16T08:16:05.383442+0000","last_fullsized":"2024-03-16T08:16:05.383442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388869+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:16:05.388869+0000","last_peered":"2024-03-16T08:16:05.388869+0000","last_clean":"2024-03-16T08:16:05.388869+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:16:05.388869+0000","last_undegraded":"2024-03-16T08:16:05.388869+0000","last_fullsized":"2024-03-16T08:16:05.388869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385161+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:16:05.385161+0000","last_peered":"2024-03-16T08:16:05.385161+0000","last_clean":"2024-03-16T08:16:05.385161+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:16:05.385161+0000","last_undegraded":"2024-03-16T08:16:05.385161+0000","last_fullsized":"2024-03-16T08:16:05.385161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383863+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:16:05.383863+0000","last_peered":"2024-03-16T08:16:05.383863+0000","last_clean":"2024-03-16T08:16:05.383863+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:16:05.383863+0000","last_undegraded":"2024-03-16T08:16:05.383863+0000","last_fullsized":"2024-03-16T08:16:05.383863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387954+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:16:05.387954+0000","last_peered":"2024-03-16T08:16:05.387954+0000","last_clean":"2024-03-16T08:16:05.387954+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:16:05.387954+0000","last_undegraded":"2024-03-16T08:16:05.387954+0000","last_fullsized":"2024-03-16T08:16:05.387954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380441+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:16:05.380441+0000","last_peered":"2024-03-16T08:16:05.380441+0000","last_clean":"2024-03-16T08:16:05.380441+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:16:05.380441+0000","last_undegraded":"2024-03-16T08:16:05.380441+0000","last_fullsized":"2024-03-16T08:16:05.380441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:15.651853+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:16:15.651853+0000","last_peered":"2024-03-16T08:16:15.651853+0000","last_clean":"2024-03-16T08:16:15.651853+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:16:15.651853+0000","last_undegraded":"2024-03-16T08:16:15.651853+0000","last_fullsized":"2024-03-16T08:16:15.651853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653236+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:16:05.653236+0000","last_peered":"2024-03-16T08:16:05.653236+0000","last_clean":"2024-03-16T08:16:05.653236+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:16:05.653236+0000","last_undegraded":"2024-03-16T08:16:05.653236+0000","last_fullsized":"2024-03-16T08:16:05.653236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381137+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:16:05.381137+0000","last_peered":"2024-03-16T08:16:05.381137+0000","last_clean":"2024-03-16T08:16:05.381137+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:16:05.381137+0000","last_undegraded":"2024-03-16T08:16:05.381137+0000","last_fullsized":"2024-03-16T08:16:05.381137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386345+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:16:05.386345+0000","last_peered":"2024-03-16T08:16:05.386345+0000","last_clean":"2024-03-16T08:16:05.386345+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:16:05.386345+0000","last_undegraded":"2024-03-16T08:16:05.386345+0000","last_fullsized":"2024-03-16T08:16:05.386345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389097+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:16:05.389097+0000","last_peered":"2024-03-16T08:16:05.389097+0000","last_clean":"2024-03-16T08:16:05.389097+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:16:05.389097+0000","last_undegraded":"2024-03-16T08:16:05.389097+0000","last_fullsized":"2024-03-16T08:16:05.389097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653199+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:16:05.653199+0000","last_peered":"2024-03-16T08:16:05.653199+0000","last_clean":"2024-03-16T08:16:05.653199+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:16:05.653199+0000","last_undegraded":"2024-03-16T08:16:05.653199+0000","last_fullsized":"2024-03-16T08:16:05.653199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":195,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:16.652963+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:16:16.652963+0000","last_peered":"2024-03-16T08:16:16.652963+0000","last_clean":"2024-03-16T08:16:16.652963+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:16:16.652963+0000","last_undegraded":"2024-03-16T08:16:16.652963+0000","last_fullsized":"2024-03-16T08:16:16.652963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":148,"num_read_kb":133,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387977+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:16:05.387977+0000","last_peered":"2024-03-16T08:16:05.387977+0000","last_clean":"2024-03-16T08:16:05.387977+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:16:05.387977+0000","last_undegraded":"2024-03-16T08:16:05.387977+0000","last_fullsized":"2024-03-16T08:16:05.387977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653483+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:16:05.653483+0000","last_peered":"2024-03-16T08:16:05.653483+0000","last_clean":"2024-03-16T08:16:05.653483+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:16:05.653483+0000","last_undegraded":"2024-03-16T08:16:05.653483+0000","last_fullsized":"2024-03-16T08:16:05.653483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385464+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:16:05.385464+0000","last_peered":"2024-03-16T08:16:05.385464+0000","last_clean":"2024-03-16T08:16:05.385464+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:16:05.385464+0000","last_undegraded":"2024-03-16T08:16:05.385464+0000","last_fullsized":"2024-03-16T08:16:05.385464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388595+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:16:05.388595+0000","last_peered":"2024-03-16T08:16:05.388595+0000","last_clean":"2024-03-16T08:16:05.388595+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:16:05.388595+0000","last_undegraded":"2024-03-16T08:16:05.388595+0000","last_fullsized":"2024-03-16T08:16:05.388595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381102+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:16:05.381102+0000","last_peered":"2024-03-16T08:16:05.381102+0000","last_clean":"2024-03-16T08:16:05.381102+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:16:05.381102+0000","last_undegraded":"2024-03-16T08:16:05.381102+0000","last_fullsized":"2024-03-16T08:16:05.381102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:14.668146+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:16:14.668146+0000","last_peered":"2024-03-16T08:16:14.668146+0000","last_clean":"2024-03-16T08:16:14.668146+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:16:14.668146+0000","last_undegraded":"2024-03-16T08:16:14.668146+0000","last_fullsized":"2024-03-16T08:16:14.668146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380746+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:16:05.380746+0000","last_peered":"2024-03-16T08:16:05.380746+0000","last_clean":"2024-03-16T08:16:05.380746+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:16:05.380746+0000","last_undegraded":"2024-03-16T08:16:05.380746+0000","last_fullsized":"2024-03-16T08:16:05.380746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387994+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:16:05.387994+0000","last_peered":"2024-03-16T08:16:05.387994+0000","last_clean":"2024-03-16T08:16:05.387994+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:16:05.387994+0000","last_undegraded":"2024-03-16T08:16:05.387994+0000","last_fullsized":"2024-03-16T08:16:05.387994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388448+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:16:05.388448+0000","last_peered":"2024-03-16T08:16:05.388448+0000","last_clean":"2024-03-16T08:16:05.388448+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:16:05.388448+0000","last_undegraded":"2024-03-16T08:16:05.388448+0000","last_fullsized":"2024-03-16T08:16:05.388448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.394865+0000","last_change":"2024-03-16T08:16:05.394457+0000","last_active":"2024-03-16T08:16:05.394865+0000","last_peered":"2024-03-16T08:16:05.394865+0000","last_clean":"2024-03-16T08:16:05.394865+0000","last_became_active":"2024-03-16T08:16:05.390239+0000","last_became_peered":"2024-03-16T08:16:05.390239+0000","last_unstale":"2024-03-16T08:16:05.394865+0000","last_undegraded":"2024-03-16T08:16:05.394865+0000","last_fullsized":"2024-03-16T08:16:05.394865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:29:29.442138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386139+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:16:05.386139+0000","last_peered":"2024-03-16T08:16:05.386139+0000","last_clean":"2024-03-16T08:16:05.386139+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:16:05.386139+0000","last_undegraded":"2024-03-16T08:16:05.386139+0000","last_fullsized":"2024-03-16T08:16:05.386139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.390744+0000","last_change":"2024-03-16T08:16:05.390744+0000","last_active":"2024-03-16T08:16:05.390744+0000","last_peered":"2024-03-16T08:16:05.390744+0000","last_clean":"2024-03-16T08:16:05.390744+0000","last_became_active":"2024-03-16T08:16:05.390378+0000","last_became_peered":"2024-03-16T08:16:05.390378+0000","last_unstale":"2024-03-16T08:16:05.390744+0000","last_undegraded":"2024-03-16T08:16:05.390744+0000","last_fullsized":"2024-03-16T08:16:05.390744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:13:29.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383634+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:16:05.383634+0000","last_peered":"2024-03-16T08:16:05.383634+0000","last_clean":"2024-03-16T08:16:05.383634+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:16:05.383634+0000","last_undegraded":"2024-03-16T08:16:05.383634+0000","last_fullsized":"2024-03-16T08:16:05.383634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380895+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:16:05.380895+0000","last_peered":"2024-03-16T08:16:05.380895+0000","last_clean":"2024-03-16T08:16:05.380895+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:16:05.380895+0000","last_undegraded":"2024-03-16T08:16:05.380895+0000","last_fullsized":"2024-03-16T08:16:05.380895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389020+0000","last_change":"2024-03-16T08:16:05.389020+0000","last_active":"2024-03-16T08:16:05.389020+0000","last_peered":"2024-03-16T08:16:05.389020+0000","last_clean":"2024-03-16T08:16:05.389020+0000","last_became_active":"2024-03-16T08:16:05.388668+0000","last_became_peered":"2024-03-16T08:16:05.388668+0000","last_unstale":"2024-03-16T08:16:05.389020+0000","last_undegraded":"2024-03-16T08:16:05.389020+0000","last_fullsized":"2024-03-16T08:16:05.389020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:34:04.661388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388171+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:16:05.388171+0000","last_peered":"2024-03-16T08:16:05.388171+0000","last_clean":"2024-03-16T08:16:05.388171+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:16:05.388171+0000","last_undegraded":"2024-03-16T08:16:05.388171+0000","last_fullsized":"2024-03-16T08:16:05.388171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:13.666580+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:16:13.666580+0000","last_peered":"2024-03-16T08:16:13.666580+0000","last_clean":"2024-03-16T08:16:13.666580+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:16:13.666580+0000","last_undegraded":"2024-03-16T08:16:13.666580+0000","last_fullsized":"2024-03-16T08:16:13.666580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387907+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:16:05.387907+0000","last_peered":"2024-03-16T08:16:05.387907+0000","last_clean":"2024-03-16T08:16:05.387907+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:16:05.387907+0000","last_undegraded":"2024-03-16T08:16:05.387907+0000","last_fullsized":"2024-03-16T08:16:05.387907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387927+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:16:05.387927+0000","last_peered":"2024-03-16T08:16:05.387927+0000","last_clean":"2024-03-16T08:16:05.387927+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:16:05.387927+0000","last_undegraded":"2024-03-16T08:16:05.387927+0000","last_fullsized":"2024-03-16T08:16:05.387927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380815+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:16:05.380815+0000","last_peered":"2024-03-16T08:16:05.380815+0000","last_clean":"2024-03-16T08:16:05.380815+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:16:05.380815+0000","last_undegraded":"2024-03-16T08:16:05.380815+0000","last_fullsized":"2024-03-16T08:16:05.380815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385044+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:16:05.385044+0000","last_peered":"2024-03-16T08:16:05.385044+0000","last_clean":"2024-03-16T08:16:05.385044+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:16:05.385044+0000","last_undegraded":"2024-03-16T08:16:05.385044+0000","last_fullsized":"2024-03-16T08:16:05.385044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380918+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:16:05.380918+0000","last_peered":"2024-03-16T08:16:05.380918+0000","last_clean":"2024-03-16T08:16:05.380918+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:16:05.380918+0000","last_undegraded":"2024-03-16T08:16:05.380918+0000","last_fullsized":"2024-03-16T08:16:05.380918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388079+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:16:05.388079+0000","last_peered":"2024-03-16T08:16:05.388079+0000","last_clean":"2024-03-16T08:16:05.388079+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:16:05.388079+0000","last_undegraded":"2024-03-16T08:16:05.388079+0000","last_fullsized":"2024-03-16T08:16:05.388079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384823+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:16:05.384823+0000","last_peered":"2024-03-16T08:16:05.384823+0000","last_clean":"2024-03-16T08:16:05.384823+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:16:05.384823+0000","last_undegraded":"2024-03-16T08:16:05.384823+0000","last_fullsized":"2024-03-16T08:16:05.384823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380801+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:16:05.380801+0000","last_peered":"2024-03-16T08:16:05.380801+0000","last_clean":"2024-03-16T08:16:05.380801+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:16:05.380801+0000","last_undegraded":"2024-03-16T08:16:05.380801+0000","last_fullsized":"2024-03-16T08:16:05.380801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387003+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:16:05.387003+0000","last_peered":"2024-03-16T08:16:05.387003+0000","last_clean":"2024-03-16T08:16:05.387003+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:16:05.387003+0000","last_undegraded":"2024-03-16T08:16:05.387003+0000","last_fullsized":"2024-03-16T08:16:05.387003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388054+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:16:05.388054+0000","last_peered":"2024-03-16T08:16:05.388054+0000","last_clean":"2024-03-16T08:16:05.388054+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:16:05.388054+0000","last_undegraded":"2024-03-16T08:16:05.388054+0000","last_fullsized":"2024-03-16T08:16:05.388054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385782+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:16:05.385782+0000","last_peered":"2024-03-16T08:16:05.385782+0000","last_clean":"2024-03-16T08:16:05.385782+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:16:05.385782+0000","last_undegraded":"2024-03-16T08:16:05.385782+0000","last_fullsized":"2024-03-16T08:16:05.385782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652991+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:16:05.652991+0000","last_peered":"2024-03-16T08:16:05.652991+0000","last_clean":"2024-03-16T08:16:05.652991+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:16:05.652991+0000","last_undegraded":"2024-03-16T08:16:05.652991+0000","last_fullsized":"2024-03-16T08:16:05.652991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387687+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:16:05.387687+0000","last_peered":"2024-03-16T08:16:05.387687+0000","last_clean":"2024-03-16T08:16:05.387687+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:16:05.387687+0000","last_undegraded":"2024-03-16T08:16:05.387687+0000","last_fullsized":"2024-03-16T08:16:05.387687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384150+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:16:05.384150+0000","last_peered":"2024-03-16T08:16:05.384150+0000","last_clean":"2024-03-16T08:16:05.384150+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:16:05.384150+0000","last_undegraded":"2024-03-16T08:16:05.384150+0000","last_fullsized":"2024-03-16T08:16:05.384150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385920+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:16:05.385920+0000","last_peered":"2024-03-16T08:16:05.385920+0000","last_clean":"2024-03-16T08:16:05.385920+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:16:05.385920+0000","last_undegraded":"2024-03-16T08:16:05.385920+0000","last_fullsized":"2024-03-16T08:16:05.385920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653334+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:16:05.653334+0000","last_peered":"2024-03-16T08:16:05.653334+0000","last_clean":"2024-03-16T08:16:05.653334+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:16:05.653334+0000","last_undegraded":"2024-03-16T08:16:05.653334+0000","last_fullsized":"2024-03-16T08:16:05.653334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384061+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:16:05.384061+0000","last_peered":"2024-03-16T08:16:05.384061+0000","last_clean":"2024-03-16T08:16:05.384061+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:16:05.384061+0000","last_undegraded":"2024-03-16T08:16:05.384061+0000","last_fullsized":"2024-03-16T08:16:05.384061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387802+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:16:05.387802+0000","last_peered":"2024-03-16T08:16:05.387802+0000","last_clean":"2024-03-16T08:16:05.387802+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:16:05.387802+0000","last_undegraded":"2024-03-16T08:16:05.387802+0000","last_fullsized":"2024-03-16T08:16:05.387802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380959+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:16:05.380959+0000","last_peered":"2024-03-16T08:16:05.380959+0000","last_clean":"2024-03-16T08:16:05.380959+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:16:05.380959+0000","last_undegraded":"2024-03-16T08:16:05.380959+0000","last_fullsized":"2024-03-16T08:16:05.380959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653465+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:16:05.653465+0000","last_peered":"2024-03-16T08:16:05.653465+0000","last_clean":"2024-03-16T08:16:05.653465+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:16:05.653465+0000","last_undegraded":"2024-03-16T08:16:05.653465+0000","last_fullsized":"2024-03-16T08:16:05.653465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381173+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:16:05.381173+0000","last_peered":"2024-03-16T08:16:05.381173+0000","last_clean":"2024-03-16T08:16:05.381173+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:16:05.381173+0000","last_undegraded":"2024-03-16T08:16:05.381173+0000","last_fullsized":"2024-03-16T08:16:05.381173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388862+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:16:05.388862+0000","last_peered":"2024-03-16T08:16:05.388862+0000","last_clean":"2024-03-16T08:16:05.388862+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:16:05.388862+0000","last_undegraded":"2024-03-16T08:16:05.388862+0000","last_fullsized":"2024-03-16T08:16:05.388862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380993+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:16:05.380993+0000","last_peered":"2024-03-16T08:16:05.380993+0000","last_clean":"2024-03-16T08:16:05.380993+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:16:05.380993+0000","last_undegraded":"2024-03-16T08:16:05.380993+0000","last_fullsized":"2024-03-16T08:16:05.380993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384729+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:16:05.384729+0000","last_peered":"2024-03-16T08:16:05.384729+0000","last_clean":"2024-03-16T08:16:05.384729+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:16:05.384729+0000","last_undegraded":"2024-03-16T08:16:05.384729+0000","last_fullsized":"2024-03-16T08:16:05.384729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381103+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:16:05.381103+0000","last_peered":"2024-03-16T08:16:05.381103+0000","last_clean":"2024-03-16T08:16:05.381103+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:16:05.381103+0000","last_undegraded":"2024-03-16T08:16:05.381103+0000","last_fullsized":"2024-03-16T08:16:05.381103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385973+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:16:05.385973+0000","last_peered":"2024-03-16T08:16:05.385973+0000","last_clean":"2024-03-16T08:16:05.385973+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:16:05.385973+0000","last_undegraded":"2024-03-16T08:16:05.385973+0000","last_fullsized":"2024-03-16T08:16:05.385973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653680+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:16:05.653680+0000","last_peered":"2024-03-16T08:16:05.653680+0000","last_clean":"2024-03-16T08:16:05.653680+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:16:05.653680+0000","last_undegraded":"2024-03-16T08:16:05.653680+0000","last_fullsized":"2024-03-16T08:16:05.653680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653237+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:16:05.653237+0000","last_peered":"2024-03-16T08:16:05.653237+0000","last_clean":"2024-03-16T08:16:05.653237+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:16:05.653237+0000","last_undegraded":"2024-03-16T08:16:05.653237+0000","last_fullsized":"2024-03-16T08:16:05.653237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384764+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:16:05.384764+0000","last_peered":"2024-03-16T08:16:05.384764+0000","last_clean":"2024-03-16T08:16:05.384764+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:16:05.384764+0000","last_undegraded":"2024-03-16T08:16:05.384764+0000","last_fullsized":"2024-03-16T08:16:05.384764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384954+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:16:05.384954+0000","last_peered":"2024-03-16T08:16:05.384954+0000","last_clean":"2024-03-16T08:16:05.384954+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:16:05.384954+0000","last_undegraded":"2024-03-16T08:16:05.384954+0000","last_fullsized":"2024-03-16T08:16:05.384954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386103+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:16:05.386103+0000","last_peered":"2024-03-16T08:16:05.386103+0000","last_clean":"2024-03-16T08:16:05.386103+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:16:05.386103+0000","last_undegraded":"2024-03-16T08:16:05.386103+0000","last_fullsized":"2024-03-16T08:16:05.386103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653614+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:16:05.653614+0000","last_peered":"2024-03-16T08:16:05.653614+0000","last_clean":"2024-03-16T08:16:05.653614+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:16:05.653614+0000","last_undegraded":"2024-03-16T08:16:05.653614+0000","last_fullsized":"2024-03-16T08:16:05.653614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.654782+0000","last_change":"2024-03-16T08:16:05.654782+0000","last_active":"2024-03-16T08:16:05.654782+0000","last_peered":"2024-03-16T08:16:05.654782+0000","last_clean":"2024-03-16T08:16:05.654782+0000","last_became_active":"2024-03-16T08:16:05.654117+0000","last_became_peered":"2024-03-16T08:16:05.654117+0000","last_unstale":"2024-03-16T08:16:05.654782+0000","last_undegraded":"2024-03-16T08:16:05.654782+0000","last_fullsized":"2024-03-16T08:16:05.654782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:17:45.678541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388078+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:16:05.388078+0000","last_peered":"2024-03-16T08:16:05.388078+0000","last_clean":"2024-03-16T08:16:05.388078+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:16:05.388078+0000","last_undegraded":"2024-03-16T08:16:05.388078+0000","last_fullsized":"2024-03-16T08:16:05.388078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384015+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:16:05.384015+0000","last_peered":"2024-03-16T08:16:05.384015+0000","last_clean":"2024-03-16T08:16:05.384015+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:16:05.384015+0000","last_undegraded":"2024-03-16T08:16:05.384015+0000","last_fullsized":"2024-03-16T08:16:05.384015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653043+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:16:05.653043+0000","last_peered":"2024-03-16T08:16:05.653043+0000","last_clean":"2024-03-16T08:16:05.653043+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:16:05.653043+0000","last_undegraded":"2024-03-16T08:16:05.653043+0000","last_fullsized":"2024-03-16T08:16:05.653043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384966+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:16:05.384966+0000","last_peered":"2024-03-16T08:16:05.384966+0000","last_clean":"2024-03-16T08:16:05.384966+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:16:05.384966+0000","last_undegraded":"2024-03-16T08:16:05.384966+0000","last_fullsized":"2024-03-16T08:16:05.384966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386061+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:16:05.386061+0000","last_peered":"2024-03-16T08:16:05.386061+0000","last_clean":"2024-03-16T08:16:05.386061+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:16:05.386061+0000","last_undegraded":"2024-03-16T08:16:05.386061+0000","last_fullsized":"2024-03-16T08:16:05.386061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":442,"num_read_kb":289,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195141,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":459215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1111720,"data_compressed":37963,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":459327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1051279,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":458818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":519657,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":460089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":521398,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":459129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1050963,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:16:21.844 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-16T08:16:21.844 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T08:16:21.844 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-16T08:16:21.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-16T08:16:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:22.515 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:22 smithi049 ceph-mon[31942]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:22.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:22 smithi129 ceph-mon[36561]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:22.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:22 smithi154 ceph-mon[36471]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:16:23.536 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:23 smithi049 ceph-mon[31942]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:23 smithi129 ceph-mon[36561]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:23.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:23 smithi154 ceph-mon[36471]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:23.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 -- 172.21.15.49:0/2001498011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f511806aa20 msgr2=0x7f511806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:23.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 --2- 172.21.15.49:0/2001498011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f511806aa20 0x7f511806aea0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51140099c0 tx=0x7f5114009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 -- 172.21.15.49:0/2001498011 shutdown_connections 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 --2- 172.21.15.49:0/2001498011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806b3e0 0x7f511810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 --2- 172.21.15.49:0/2001498011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5118108300 0x7f51181086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 --2- 172.21.15.49:0/2001498011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f511806aa20 0x7f511806aea0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 -- 172.21.15.49:0/2001498011 >> 172.21.15.49:0/2001498011 conn(0x7f51180781e0 msgr2=0x7f51180785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 -- 172.21.15.49:0/2001498011 shutdown_connections 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 -- 172.21.15.49:0/2001498011 wait complete. 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.761+0000 7f511f697700 1 Processor -- start 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 -- start start 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 0x7f511819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 0x7f5118198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51181125a0 con 0x7f5118108300 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5118112420 con 0x7f511806aa20 2024-03-16T08:16:23.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511f697700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5118112720 con 0x7f511806b3e0 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511dc34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 0x7f5118198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511cc32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 0x7f511819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511dc34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 0x7f5118198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35818/0 (socket says 172.21.15.49:35818) 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511cc32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 0x7f511819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:35452/0 (socket says 172.21.15.49:35452) 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511dc34700 1 -- 172.21.15.49:0/1972077053 learned_addr learned my addr 172.21.15.49:0/1972077053 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:57104/0 (socket says 172.21.15.49:57104) 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 -- 172.21.15.49:0/1972077053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 msgr2=0x7f511819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 0x7f511819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 -- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 msgr2=0x7f5118198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 0x7f5118198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 -- 172.21.15.49:0/1972077053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5118198fe0 con 0x7f511806aa20 2024-03-16T08:16:23.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.762+0000 7f511d433700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:23.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.763+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c004220 con 0x7f511806aa20 2024-03-16T08:16:23.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.763+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51181992d0 con 0x7f511806aa20 2024-03-16T08:16:23.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.763+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f510c004d10 con 0x7f511806aa20 2024-03-16T08:16:23.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.763+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c005020 con 0x7f511806aa20 2024-03-16T08:16:23.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.763+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181011b0 con 0x7f511806aa20 2024-03-16T08:16:23.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.764+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51181039b0 con 0x7f511806aa20 2024-03-16T08:16:23.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.765+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f510c004750 con 0x7f511806aa20 2024-03-16T08:16:23.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.766+0000 7f510a7fc700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 0x7f51040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:23.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.766+0000 7f511cc32700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 0x7f51040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:23.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.766+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f510c0cebc0 con 0x7f511806aa20 2024-03-16T08:16:23.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.766+0000 7f511cc32700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 0x7f51040a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51180787e0 tx=0x7f5114005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:23.774+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f510c09bb60 con 0x7f511806aa20 2024-03-16T08:16:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.024+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5118199d30 con 0x7f511806aa20 2024-03-16T08:16:24.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.024+0000 7f510a7fc700 1 -- 172.21.15.49:0/1972077053 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f510c09b460 con 0x7f511806aa20 2024-03-16T08:16:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:24.025 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.026+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 msgr2=0x7f51040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.026+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 0x7f51040a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51180787e0 tx=0x7f5114005fb0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.026+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 msgr2=0x7f511819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.026+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c00ed00 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 shutdown_connections 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f51040a2920 0x7f51040a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5118108300 0x7f5118198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f511806aa20 0x7f511819e680 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 --2- 172.21.15.49:0/1972077053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f511806b3e0 0x7f511819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 >> 172.21.15.49:0/1972077053 conn(0x7f51180781e0 msgr2=0x7f51180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 shutdown_connections 2024-03-16T08:16:24.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.027+0000 7f511f697700 1 -- 172.21.15.49:0/1972077053 wait complete. 2024-03-16T08:16:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:24 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1972077053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T08:16:24.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:24 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1972077053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T08:16:24.683 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-16T08:16:24.683 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:24 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1972077053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T08:16:24.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/505015718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481018f0 msgr2=0x7fbd48101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/505015718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481018f0 0x7fbd48101d70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009900 tx=0x7fbd3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:24.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/505015718 shutdown_connections 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/505015718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481022b0 0x7fbd48108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/505015718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481018f0 0x7fbd48101d70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/505015718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/505015718 >> 172.21.15.49:0/505015718 conn(0x7fbd480fbc70 msgr2=0x7fbd480fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/505015718 shutdown_connections 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/505015718 wait complete. 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.797+0000 7fbd4d4dc640 1 Processor -- start 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 -- start start 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481018f0 0x7fbd48198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810d4a0 con 0x7fbd481022b0 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810d320 con 0x7fbd48100700 2024-03-16T08:16:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd4d4dc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810d620 con 0x7fbd481018f0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35840/0 (socket says 172.21.15.49:35840) 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 -- 172.21.15.49:0/3724749252 learned_addr learned my addr 172.21.15.49:0/3724749252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd467fc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481018f0 0x7fbd48198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 -- 172.21.15.49:0/3724749252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481018f0 msgr2=0x7fbd48198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481018f0 0x7fbd48198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 -- 172.21.15.49:0/3724749252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 msgr2=0x7fbd48197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 -- 172.21.15.49:0/3724749252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4819f740 con 0x7fbd481022b0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd46ffd640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.798+0000 7fbd477fe640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e970 tx=0x7fbd3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.799+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3800ccc0 con 0x7fbd481022b0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.799+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4819fa30 con 0x7fbd481022b0 2024-03-16T08:16:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.799+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3800ce60 con 0x7fbd481022b0 2024-03-16T08:16:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.799+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38010450 con 0x7fbd481022b0 2024-03-16T08:16:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.799+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4819ffa0 con 0x7fbd481022b0 2024-03-16T08:16:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.800+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd48107a20 con 0x7fbd481022b0 2024-03-16T08:16:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.802+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fbd38005020 con 0x7fbd481022b0 2024-03-16T08:16:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.802+0000 7fbd3ffff640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 0x7fbd340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.802+0000 7fbd46ffd640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 0x7fbd340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.802+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fbd380cefb0 con 0x7fbd481022b0 2024-03-16T08:16:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.803+0000 7fbd46ffd640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 0x7fbd340a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd48101750 tx=0x7fbd40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:24.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.807+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd3809bf20 con 0x7fbd481022b0 2024-03-16T08:16:24.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.994+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd4802d3d0 con 0x7fbd481022b0 2024-03-16T08:16:24.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.995+0000 7fbd3ffff640 1 -- 172.21.15.49:0/3724749252 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fbd3809b820 con 0x7fbd481022b0 2024-03-16T08:16:24.996 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:24.996 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 msgr2=0x7fbd340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 0x7fbd340a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd48101750 tx=0x7fbd40009500 comp rx=0 tx=0).stop 2024-03-16T08:16:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 msgr2=0x7fbd4819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e970 tx=0x7fbd3800ed70 comp rx=0 tx=0).stop 2024-03-16T08:16:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 shutdown_connections 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fbd340a2890 0x7fbd340a4d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd481022b0 0x7fbd4819f0d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd481018f0 0x7fbd48198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 --2- 172.21.15.49:0/3724749252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd48100700 0x7fbd48197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 >> 172.21.15.49:0/3724749252 conn(0x7fbd480fbc70 msgr2=0x7fbd48104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 shutdown_connections 2024-03-16T08:16:24.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:24.998+0000 7fbd4d4dc640 1 -- 172.21.15.49:0/3724749252 wait complete. 2024-03-16T08:16:25.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:16:25.010 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:16:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 -- 172.21.15.49:0/3187227442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c104c30 msgr2=0x7f448c107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 --2- 172.21.15.49:0/3187227442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c104c30 0x7f448c107020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f447c009900 tx=0x7f447c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 -- 172.21.15.49:0/3187227442 shutdown_connections 2024-03-16T08:16:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 --2- 172.21.15.49:0/3187227442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c107560 0x7f448c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 --2- 172.21.15.49:0/3187227442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c104c30 0x7f448c107020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 --2- 172.21.15.49:0/3187227442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 -- 172.21.15.49:0/3187227442 >> 172.21.15.49:0/3187227442 conn(0x7f448c0fbc70 msgr2=0x7f448c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 -- 172.21.15.49:0/3187227442 shutdown_connections 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.122+0000 7f4491b37640 1 -- 172.21.15.49:0/3187227442 wait complete. 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 Processor -- start 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 -- start start 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 0x7f448c1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f448c10d460 con 0x7f448c107560 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f448c10d2e0 con 0x7f448c102300 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f4491b37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f448c10d5e0 con 0x7f448c104c30 2024-03-16T08:16:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35858/0 (socket says 172.21.15.49:35858) 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 0x7f448c1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 -- 172.21.15.49:0/2049656353 learned_addr learned my addr 172.21.15.49:0/2049656353 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448b7fe640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 -- 172.21.15.49:0/2049656353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 msgr2=0x7f448c1002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 0x7f448c1002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 -- 172.21.15.49:0/2049656353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 msgr2=0x7f448c101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448bfff640 1 -- 172.21.15.49:0/2049656353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c101080 con 0x7f448c107560 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448b7fe640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.123+0000 7f448affd640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 0x7f448c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f448bfff640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f448400eb10 tx=0x7f448400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484004b20 con 0x7f448c107560 2024-03-16T08:16:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c1a1c50 con 0x7f448c107560 2024-03-16T08:16:25.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4484004cc0 con 0x7f448c107560 2024-03-16T08:16:25.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484010760 con 0x7f448c107560 2024-03-16T08:16:25.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.124+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c1a2170 con 0x7f448c107560 2024-03-16T08:16:25.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.125+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c0685d0 con 0x7f448c107560 2024-03-16T08:16:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.126+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4484005000 con 0x7f448c107560 2024-03-16T08:16:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.127+0000 7f4488ff9640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 0x7f44740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.127+0000 7f448b7fe640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 0x7f44740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.127+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f44840cf640 con 0x7f448c107560 2024-03-16T08:16:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.127+0000 7f448b7fe640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 0x7f44740a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44800095f0 tx=0x7f4480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.130+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f448409c5b0 con 0x7f448c107560 2024-03-16T08:16:25.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.313+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f448c02d1e0 con 0x7f448c107560 2024-03-16T08:16:25.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.314+0000 7f4488ff9640 1 -- 172.21.15.49:0/2049656353 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f448409beb0 con 0x7f448c107560 2024-03-16T08:16:25.315 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 msgr2=0x7f44740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 0x7f44740a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44800095f0 tx=0x7f4480005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 msgr2=0x7f448c1007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f448400eb10 tx=0x7f448400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 shutdown_connections 2024-03-16T08:16:25.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f44740a2890 0x7f44740a4d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c107560 0x7f448c1007f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f448c104c30 0x7f448c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 --2- 172.21.15.49:0/2049656353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c102300 0x7f448c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 >> 172.21.15.49:0/2049656353 conn(0x7f448c0fbc70 msgr2=0x7f448c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 shutdown_connections 2024-03-16T08:16:25.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.316+0000 7f4491b37640 1 -- 172.21.15.49:0/2049656353 wait complete. 2024-03-16T08:16:25.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:16:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- 172.21.15.49:0/3709890891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 msgr2=0x7f9054101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 --2- 172.21.15.49:0/3709890891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054101c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9044009ca0 tx=0x7f904402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- 172.21.15.49:0/3709890891 shutdown_connections 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 --2- 172.21.15.49:0/3709890891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f9054109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 --2- 172.21.15.49:0/3709890891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 0x7f9054102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 --2- 172.21.15.49:0/3709890891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054101c20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- 172.21.15.49:0/3709890891 >> 172.21.15.49:0/3709890891 conn(0x7f90540fcd70 msgr2=0x7f90540ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- 172.21.15.49:0/3709890891 shutdown_connections 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- 172.21.15.49:0/3709890891 wait complete. 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 Processor -- start 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.439+0000 7f905c473640 1 -- start start 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 0x7f9054198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f905419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d470 con 0x7f9054101800 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d2f0 con 0x7f90541033b0 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d5f0 con 0x7f90541029f0 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35880/0 (socket says 172.21.15.49:35880) 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 -- 172.21.15.49:0/1483514674 learned_addr learned my addr 172.21.15.49:0/1483514674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f9059a0e640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 0x7f9054198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905aa10640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f905419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 -- 172.21.15.49:0/1483514674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 msgr2=0x7f9054198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 0x7f9054198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 -- 172.21.15.49:0/1483514674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 msgr2=0x7f905419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f905419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 -- 172.21.15.49:0/1483514674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905419f750 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905aa10640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f905419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905a20f640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f90440324c0 tx=0x7f9044032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044041020 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044034cd0 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905419f9e0 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044039040 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.440+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905419fed0 con 0x7f9054101800 2024-03-16T08:16:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.441+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9054108bf0 con 0x7f9054101800 2024-03-16T08:16:25.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.442+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9044044020 con 0x7f9054101800 2024-03-16T08:16:25.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.443+0000 7f90537fe640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 0x7f903c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.443+0000 7f9059a0e640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 0x7f903c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.443+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f90440f0a30 con 0x7f9054101800 2024-03-16T08:16:25.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.444+0000 7f9059a0e640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 0x7f903c0a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9054199040 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.448+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f904404b020 con 0x7f9054101800 2024-03-16T08:16:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:25 smithi129 ceph-mon[36561]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3724749252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2049656353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:25.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.630+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f905402d400 con 0x7f9054101800 2024-03-16T08:16:25.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.630+0000 7f90537fe640 1 -- 172.21.15.49:0/1483514674 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f90440bdb40 con 0x7f9054101800 2024-03-16T08:16:25.631 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 msgr2=0x7f903c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 0x7f903c0a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9054199040 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 msgr2=0x7f9054197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f90440324c0 tx=0x7f9044032f40 comp rx=0 tx=0).stop 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 shutdown_connections 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90541033b0 0x7f905419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90541029f0 0x7f9054198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f903c0a2890 0x7f903c0a4d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 --2- 172.21.15.49:0/1483514674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054101800 0x7f9054197b30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 >> 172.21.15.49:0/1483514674 conn(0x7f90540fcd70 msgr2=0x7f9054105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 shutdown_connections 2024-03-16T08:16:25.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.632+0000 7f905c473640 1 -- 172.21.15.49:0/1483514674 wait complete. 2024-03-16T08:16:25.643 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:25 smithi049 ceph-mon[31942]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:25.643 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3724749252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:25.643 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2049656353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:25.644 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:16:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:25 smithi154 ceph-mon[36471]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3724749252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:25.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2049656353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.755+0000 7ff6243c6640 1 -- 172.21.15.49:0/1472414032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 msgr2=0x7ff61c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.755+0000 7ff6243c6640 1 --2- 172.21.15.49:0/1472414032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c101d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009900 tx=0x7ff60c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- 172.21.15.49:0/1472414032 shutdown_connections 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- 172.21.15.49:0/1472414032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c102290 0x7ff61c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- 172.21.15.49:0/1472414032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c101d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- 172.21.15.49:0/1472414032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c1006e0 0x7ff61c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- 172.21.15.49:0/1472414032 >> 172.21.15.49:0/1472414032 conn(0x7ff61c0fbc70 msgr2=0x7ff61c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- 172.21.15.49:0/1472414032 shutdown_connections 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- 172.21.15.49:0/1472414032 wait complete. 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 Processor -- start 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- start start 2024-03-16T08:16:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 0x7ff61c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c102290 0x7ff61c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10c360 con 0x7ff61c1018d0 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10c1e0 con 0x7ff61c1006e0 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff6243c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10c4e0 con 0x7ff61c102290 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff621961640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff621961640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35898/0 (socket says 172.21.15.49:35898) 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff622162640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 0x7ff61c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.756+0000 7ff621961640 1 -- 172.21.15.49:0/3312788432 learned_addr learned my addr 172.21.15.49:0/3312788432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff622963640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c102290 0x7ff61c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 -- 172.21.15.49:0/3312788432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c102290 msgr2=0x7ff61c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c102290 0x7ff61c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 -- 172.21.15.49:0/3312788432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 msgr2=0x7ff61c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 0x7ff61c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 -- 172.21.15.49:0/3312788432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c19f7a0 con 0x7ff61c1018d0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff622162640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 0x7ff61c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff621961640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009230 tx=0x7ff60c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c041070 con 0x7ff61c1018d0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c19fa30 con 0x7ff61c1018d0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60c004450 con 0x7ff61c1018d0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c003e20 con 0x7ff61c1018d0 2024-03-16T08:16:25.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.757+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c19ff20 con 0x7ff61c1018d0 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.758+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c1a00f0 con 0x7ff61c1018d0 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.759+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff60c044070 con 0x7ff61c1018d0 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.760+0000 7ff6137fe640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 0x7ff6080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.760+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff60c0f0640 con 0x7ff61c1018d0 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.760+0000 7ff622162640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 0x7ff6080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:25.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.760+0000 7ff622162640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 0x7ff6080a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff61c101730 tx=0x7ff614005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:25.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.764+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff60c0bd070 con 0x7ff61c1018d0 2024-03-16T08:16:25.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.947+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff61c1a0480 con 0x7ff61c1018d0 2024-03-16T08:16:25.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.947+0000 7ff6137fe640 1 -- 172.21.15.49:0/3312788432 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff60c0c1110 con 0x7ff61c1018d0 2024-03-16T08:16:25.948 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.949+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 msgr2=0x7ff6080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 0x7ff6080a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff61c101730 tx=0x7ff614005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 msgr2=0x7ff61c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009230 tx=0x7ff60c02c650 comp rx=0 tx=0).stop 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 shutdown_connections 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff61c102290 0x7ff61c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff6080a2890 0x7ff6080a4d50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff61c1018d0 0x7ff61c1980c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 --2- 172.21.15.49:0/3312788432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff61c1006e0 0x7ff61c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 >> 172.21.15.49:0/3312788432 conn(0x7ff61c0fbc70 msgr2=0x7ff61c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 shutdown_connections 2024-03-16T08:16:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:25.950+0000 7ff6243c6640 1 -- 172.21.15.49:0/3312788432 wait complete. 2024-03-16T08:16:25.961 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:16:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 -- 172.21.15.49:0/1075871185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 msgr2=0x7fe5901099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1075871185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe5901099b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ca0 tx=0x7fe58802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 -- 172.21.15.49:0/1075871185 shutdown_connections 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1075871185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe5901099b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1075871185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 0x7fe590107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1075871185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 0x7fe5901046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 -- 172.21.15.49:0/1075871185 >> 172.21.15.49:0/1075871185 conn(0x7fe5900fbd10 msgr2=0x7fe5900fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 -- 172.21.15.49:0/1075871185 shutdown_connections 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.069+0000 7fe5957e1640 1 -- 172.21.15.49:0/1075871185 wait complete. 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 Processor -- start 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 -- start start 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 0x7fe590197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 0x7fe590198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010d490 con 0x7fe590107560 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010d310 con 0x7fe590102300 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe5957e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010d610 con 0x7fe590104c30 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe587fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 0x7fe590198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35924/0 (socket says 172.21.15.49:35924) 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 -- 172.21.15.49:0/1249468917 learned_addr learned my addr 172.21.15.49:0/1249468917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58effd640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 0x7fe590197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 -- 172.21.15.49:0/1249468917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 msgr2=0x7fe590198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 0x7fe590198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 -- 172.21.15.49:0/1249468917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 msgr2=0x7fe590197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 0x7fe590197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 -- 172.21.15.49:0/1249468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59019f6e0 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.070+0000 7fe58f7fe640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe588037040 tx=0x7fe588032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.071+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588041020 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.071+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588034cd0 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.071+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59019f970 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.071+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588039040 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.071+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59019feb0 con 0x7fe590107560 2024-03-16T08:16:26.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.072+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900fd800 con 0x7fe590107560 2024-03-16T08:16:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.073+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe588044020 con 0x7fe590107560 2024-03-16T08:16:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.073+0000 7fe58cff9640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 0x7fe57c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.074+0000 7fe58effd640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 0x7fe57c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.074+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe5880f08a0 con 0x7fe590107560 2024-03-16T08:16:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.074+0000 7fe58effd640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 0x7fe57c0a4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe580009bd0 tx=0x7fe580005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.078+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe58804b020 con 0x7fe590107560 2024-03-16T08:16:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.303+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5900623e0 con 0x7fe590107560 2024-03-16T08:16:26.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.304+0000 7fe58cff9640 1 -- 172.21.15.49:0/1249468917 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7fe5880b7e20 con 0x7fe590107560 2024-03-16T08:16:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:26.305 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:16:26.305 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 msgr2=0x7fe57c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 0x7fe57c0a4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe580009bd0 tx=0x7fe580005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 msgr2=0x7fe59019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe588037040 tx=0x7fe588032f40 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 shutdown_connections 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe57c0a2910 0x7fe57c0a4dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe590107560 0x7fe59019f070 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe590104c30 0x7fe590198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 --2- 172.21.15.49:0/1249468917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe590102300 0x7fe590197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.306+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 >> 172.21.15.49:0/1249468917 conn(0x7fe5900fbd10 msgr2=0x7fe590102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.307+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 shutdown_connections 2024-03-16T08:16:26.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.307+0000 7fe5957e1640 1 -- 172.21.15.49:0/1249468917 wait complete. 2024-03-16T08:16:26.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:26.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.427+0000 7fd863827640 1 -- 172.21.15.49:0/1089797843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c104c10 msgr2=0x7fd85c107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.427+0000 7fd863827640 1 --2- 172.21.15.49:0/1089797843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c104c10 0x7fd85c107000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd850009860 tx=0x7fd85002bb70 comp rx=0 tx=0).stop 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- 172.21.15.49:0/1089797843 shutdown_connections 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- 172.21.15.49:0/1089797843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c107540 0x7fd85c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- 172.21.15.49:0/1089797843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c104c10 0x7fd85c107000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- 172.21.15.49:0/1089797843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c1022e0 0x7fd85c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- 172.21.15.49:0/1089797843 >> 172.21.15.49:0/1089797843 conn(0x7fd85c0fbc70 msgr2=0x7fd85c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- 172.21.15.49:0/1089797843 shutdown_connections 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- 172.21.15.49:0/1089797843 wait complete. 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 Processor -- start 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- start start 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c1022e0 0x7fd85c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c104c10 0x7fd85c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c10d480 con 0x7fd85c107540 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c10d300 con 0x7fd85c104c10 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.428+0000 7fd863827640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c10d600 con 0x7fd85c1022e0 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35946/0 (socket says 172.21.15.49:35946) 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 -- 172.21.15.49:0/1570699641 learned_addr learned my addr 172.21.15.49:0/1570699641 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd8615c3640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c1022e0 0x7fd85c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd860dc2640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c104c10 0x7fd85c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 -- 172.21.15.49:0/1570699641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c1022e0 msgr2=0x7fd85c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c1022e0 0x7fd85c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 -- 172.21.15.49:0/1570699641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c104c10 msgr2=0x7fd85c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c104c10 0x7fd85c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 -- 172.21.15.49:0/1570699641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c19f770 con 0x7fd85c107540 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd861dc4640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00eb10 tx=0x7fd84c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c004b20 con 0x7fd85c107540 2024-03-16T08:16:26.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c19fa60 con 0x7fd85c107540 2024-03-16T08:16:26.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c004cc0 con 0x7fd85c107540 2024-03-16T08:16:26.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c010760 con 0x7fd85c107540 2024-03-16T08:16:26.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.429+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c19ff80 con 0x7fd85c107540 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.430+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c0fd760 con 0x7fd85c107540 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.431+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd84c005020 con 0x7fd85c107540 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.432+0000 7fd85a7fc640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 0x7fd8440a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.432+0000 7fd8615c3640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 0x7fd8440a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.432+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fd84c0cf2a0 con 0x7fd85c107540 2024-03-16T08:16:26.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.432+0000 7fd8615c3640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 0x7fd8440a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd854009530 tx=0x7fd854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.436+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd84c09c210 con 0x7fd85c107540 2024-03-16T08:16:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:26 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1483514674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:26 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3312788432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:26 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1249468917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:26.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.621+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd85c02d550 con 0x7fd85c107540 2024-03-16T08:16:26.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.623+0000 7fd85a7fc640 1 -- 172.21.15.49:0/1570699641 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fd84c09bb10 con 0x7fd85c107540 2024-03-16T08:16:26.623 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:26.623 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.624+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 msgr2=0x7fd8440a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.624+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 0x7fd8440a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd854009530 tx=0x7fd854009260 comp rx=0 tx=0).stop 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.624+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 msgr2=0x7fd85c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.624+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00eb10 tx=0x7fd84c00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 shutdown_connections 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd8440a2890 0x7fd8440a4d50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd85c107540 0x7fd85c19f100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd85c104c10 0x7fd85c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 --2- 172.21.15.49:0/1570699641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c1022e0 0x7fd85c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 >> 172.21.15.49:0/1570699641 conn(0x7fd85c0fbc70 msgr2=0x7fd85c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 shutdown_connections 2024-03-16T08:16:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.625+0000 7fd863827640 1 -- 172.21.15.49:0/1570699641 wait complete. 2024-03-16T08:16:26.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:26 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1483514674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:26 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3312788432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:26 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1249468917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:26.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:16:26.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:16:26.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:26 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1483514674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:26 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3312788432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:26.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:26 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1249468917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 -- 172.21.15.49:0/1930426833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1033a0 msgr2=0x7f938c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 --2- 172.21.15.49:0/1930426833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1033a0 0x7f938c109c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9384009860 tx=0x7f938402bb70 comp rx=0 tx=0).stop 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 -- 172.21.15.49:0/1930426833 shutdown_connections 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 --2- 172.21.15.49:0/1930426833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1033a0 0x7f938c109c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 --2- 172.21.15.49:0/1930426833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1029e0 0x7f938c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 --2- 172.21.15.49:0/1930426833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1017f0 0x7f938c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.747+0000 7f9391dd4640 1 -- 172.21.15.49:0/1930426833 >> 172.21.15.49:0/1930426833 conn(0x7f938c0fcd60 msgr2=0x7f938c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- 172.21.15.49:0/1930426833 shutdown_connections 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- 172.21.15.49:0/1930426833 wait complete. 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 Processor -- start 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- start start 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1029e0 0x7f938c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1033a0 0x7f938c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10d450 con 0x7f938c1017f0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10d2d0 con 0x7f938c1033a0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.748+0000 7f9391dd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10d5d0 con 0x7f938c1029e0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35970/0 (socket says 172.21.15.49:35970) 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 -- 172.21.15.49:0/527059563 learned_addr learned my addr 172.21.15.49:0/527059563 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938affd640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1029e0 0x7f938c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938bfff640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1033a0 0x7f938c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 -- 172.21.15.49:0/527059563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1029e0 msgr2=0x7f938c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1029e0 0x7f938c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 -- 172.21.15.49:0/527059563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1033a0 msgr2=0x7f938c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1033a0 0x7f938c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 -- 172.21.15.49:0/527059563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c19f6c0 con 0x7f938c1017f0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f938b7fe640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f937c00c3f0 tx=0x7f937c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c01b070 con 0x7f938c1017f0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f937c00bb10 con 0x7f938c1017f0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c19f9b0 con 0x7f938c1017f0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c016d80 con 0x7f938c1017f0 2024-03-16T08:16:26.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.749+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c19ff00 con 0x7f938c1017f0 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.750+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c108bb0 con 0x7f938c1017f0 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.751+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f937c01e070 con 0x7f938c1017f0 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.752+0000 7f9388ff9640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 0x7f93780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.752+0000 7f938affd640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 0x7f93780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.752+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f937c0cea60 con 0x7f938c1017f0 2024-03-16T08:16:26.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.752+0000 7f938affd640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 0x7f93780a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f938c198fb0 tx=0x7f9374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.756+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f937c09b380 con 0x7f938c1017f0 2024-03-16T08:16:26.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.943+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f938c02d1b0 con 0x7f938c1017f0 2024-03-16T08:16:26.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.944+0000 7f9388ff9640 1 -- 172.21.15.49:0/527059563 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f938c02d1b0 con 0x7f938c1017f0 2024-03-16T08:16:26.944 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 msgr2=0x7f93780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 0x7f93780a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f938c198fb0 tx=0x7f9374005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 msgr2=0x7f938c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f937c00c3f0 tx=0x7f937c009f90 comp rx=0 tx=0).stop 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 shutdown_connections 2024-03-16T08:16:26.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f938c1033a0 0x7f938c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938c1029e0 0x7f938c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f93780a2890 0x7f93780a4d50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 --2- 172.21.15.49:0/527059563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f938c1017f0 0x7f938c197aa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 >> 172.21.15.49:0/527059563 conn(0x7f938c0fcd60 msgr2=0x7f938c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 shutdown_connections 2024-03-16T08:16:26.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:26.946+0000 7f9391dd4640 1 -- 172.21.15.49:0/527059563 wait complete. 2024-03-16T08:16:26.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:16:27.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 -- 172.21.15.49:0/805390708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c104c10 msgr2=0x7ff46c107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 --2- 172.21.15.49:0/805390708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c104c10 0x7ff46c107000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 -- 172.21.15.49:0/805390708 shutdown_connections 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 --2- 172.21.15.49:0/805390708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 0x7ff46c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 --2- 172.21.15.49:0/805390708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c104c10 0x7ff46c107000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 --2- 172.21.15.49:0/805390708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c1022e0 0x7ff46c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 -- 172.21.15.49:0/805390708 >> 172.21.15.49:0/805390708 conn(0x7ff46c0fbc90 msgr2=0x7ff46c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 -- 172.21.15.49:0/805390708 shutdown_connections 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.068+0000 7ff47453b640 1 -- 172.21.15.49:0/805390708 wait complete. 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 Processor -- start 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 -- start start 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c104c10 0x7ff46c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 0x7ff46c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d480 con 0x7ff46c1022e0 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d300 con 0x7ff46c104c10 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff47453b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d600 con 0x7ff46c107540 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35990/0 (socket says 172.21.15.49:35990) 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 -- 172.21.15.49:0/1311635830 learned_addr learned my addr 172.21.15.49:0/1311635830 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff471ad6640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c104c10 0x7ff46c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff472ad8640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 0x7ff46c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 -- 172.21.15.49:0/1311635830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 msgr2=0x7ff46c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 0x7ff46c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 -- 172.21.15.49:0/1311635830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c104c10 msgr2=0x7ff46c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c104c10 0x7ff46c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.069+0000 7ff4722d7640 1 -- 172.21.15.49:0/1311635830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c19f770 con 0x7ff46c1022e0 2024-03-16T08:16:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff4722d7640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ea60 tx=0x7ff46800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46800cde0 con 0x7ff46c1022e0 2024-03-16T08:16:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4680035f0 con 0x7ff46c1022e0 2024-03-16T08:16:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c19fa60 con 0x7ff46c1022e0 2024-03-16T08:16:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468010670 con 0x7ff46c1022e0 2024-03-16T08:16:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.070+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c19ffd0 con 0x7ff46c1022e0 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.071+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c0fd780 con 0x7ff46c1022e0 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.072+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff4680037a0 con 0x7ff46c1022e0 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.072+0000 7ff4637fe640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 0x7ff4580a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.073+0000 7ff471ad6640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 0x7ff4580a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.073+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff4680cee20 con 0x7ff46c1022e0 2024-03-16T08:16:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.073+0000 7ff471ad6640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 0x7ff4580a4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000c00 tx=0x7ff45c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.077+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff46809bd90 con 0x7ff46c1022e0 2024-03-16T08:16:27.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.259+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff46c0623e0 con 0x7ff46c1022e0 2024-03-16T08:16:27.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.260+0000 7ff4637fe640 1 -- 172.21.15.49:0/1311635830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ff46809b690 con 0x7ff46c1022e0 2024-03-16T08:16:27.260 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 msgr2=0x7ff4580a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 0x7ff4580a4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000c00 tx=0x7ff45c005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 msgr2=0x7ff46c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ea60 tx=0x7ff46800ef20 comp rx=0 tx=0).stop 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 shutdown_connections 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46c107540 0x7ff46c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff46c104c10 0x7ff46c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4580a2910 0x7ff4580a4dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 --2- 172.21.15.49:0/1311635830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff46c1022e0 0x7ff46c197b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 >> 172.21.15.49:0/1311635830 conn(0x7ff46c0fbc90 msgr2=0x7ff46c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 shutdown_connections 2024-03-16T08:16:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.262+0000 7ff47453b640 1 -- 172.21.15.49:0/1311635830 wait complete. 2024-03-16T08:16:27.273 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:16:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 -- 172.21.15.49:0/1754722406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5201029f0 msgr2=0x7fe520102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 --2- 172.21.15.49:0/1754722406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5201029f0 0x7fe520102e70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 -- 172.21.15.49:0/1754722406 shutdown_connections 2024-03-16T08:16:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 --2- 172.21.15.49:0/1754722406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 0x7fe520109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 --2- 172.21.15.49:0/1754722406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5201029f0 0x7fe520102e70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 --2- 172.21.15.49:0/1754722406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe520101800 0x7fe520101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 -- 172.21.15.49:0/1754722406 >> 172.21.15.49:0/1754722406 conn(0x7fe5200fcd70 msgr2=0x7fe5200ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 -- 172.21.15.49:0/1754722406 shutdown_connections 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.385+0000 7fe524b32640 1 -- 172.21.15.49:0/1754722406 wait complete. 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 Processor -- start 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 -- start start 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5201029f0 0x7fe520198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 0x7fe52019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d480 con 0x7fe520101800 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d300 con 0x7fe5201033b0 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe524b32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d600 con 0x7fe5201029f0 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36018/0 (socket says 172.21.15.49:36018) 2024-03-16T08:16:27.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 -- 172.21.15.49:0/2274486851 learned_addr learned my addr 172.21.15.49:0/2274486851 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51ed9d640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 0x7fe52019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51dd9b640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5201029f0 0x7fe520198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 -- 172.21.15.49:0/2274486851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5201029f0 msgr2=0x7fe520198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5201029f0 0x7fe520198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 -- 172.21.15.49:0/2274486851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 msgr2=0x7fe52019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 0x7fe52019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 -- 172.21.15.49:0/2274486851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52019f710 con 0x7fe520101800 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.386+0000 7fe51e59c640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e9c0 tx=0x7fe51000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.387+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000cd80 con 0x7fe520101800 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.387+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5100035f0 con 0x7fe520101800 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.387+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510010690 con 0x7fe520101800 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.387+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52019fa00 con 0x7fe520101800 2024-03-16T08:16:27.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.387+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52019fed0 con 0x7fe520101800 2024-03-16T08:16:27.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.388+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe520108b20 con 0x7fe520101800 2024-03-16T08:16:27.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.389+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe5100037a0 con 0x7fe520101800 2024-03-16T08:16:27.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.390+0000 7fe50f7fe640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 0x7fe5080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.390+0000 7fe51dd9b640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 0x7fe5080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.390+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe5100cef20 con 0x7fe520101800 2024-03-16T08:16:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.390+0000 7fe51dd9b640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 0x7fe5080a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c7b0 tx=0x7fe514005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.394+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe51009be90 con 0x7fe520101800 2024-03-16T08:16:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:27 smithi049 ceph-mon[31942]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:27 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1570699641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:27 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/527059563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:27 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1311635830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.578+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe520062480 con 0x7fe520101800 2024-03-16T08:16:27.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.578+0000 7fe50f7fe640 1 -- 172.21.15.49:0/2274486851 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fe51009b790 con 0x7fe520101800 2024-03-16T08:16:27.578 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:16:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 msgr2=0x7fe5080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 0x7fe5080a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c7b0 tx=0x7fe514005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 msgr2=0x7fe520197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e9c0 tx=0x7fe51000ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 shutdown_connections 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5201033b0 0x7fe52019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5201029f0 0x7fe520198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5080a2890 0x7fe5080a4d50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 --2- 172.21.15.49:0/2274486851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe520101800 0x7fe520197af0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 >> 172.21.15.49:0/2274486851 conn(0x7fe5200fcd70 msgr2=0x7fe520105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 shutdown_connections 2024-03-16T08:16:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.580+0000 7fe524b32640 1 -- 172.21.15.49:0/2274486851 wait complete. 2024-03-16T08:16:27.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:16:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:27 smithi129 ceph-mon[36561]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:27 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1570699641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:27 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/527059563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:27 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1311635830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2945006912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b481033c0 msgr2=0x7f1b48109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2945006912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b481033c0 0x7f1b48109c50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009900 tx=0x7f1b3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2945006912 shutdown_connections 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2945006912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b481033c0 0x7f1b48109c50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2945006912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48102a00 0x7f1b48102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2945006912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b48101810 0x7f1b48101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2945006912 >> 172.21.15.49:0/2945006912 conn(0x7f1b480fcdc0 msgr2=0x7f1b480ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2945006912 shutdown_connections 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.699+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2945006912 wait complete. 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 Processor -- start 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 -- start start 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48101810 0x7f1b48197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 0x7f1b4819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810d490 con 0x7f1b48102a00 2024-03-16T08:16:27.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810d310 con 0x7f1b481033c0 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4eae8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810d610 con 0x7f1b48101810 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36044/0 (socket says 172.21.15.49:36044) 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 -- 172.21.15.49:0/2784644891 learned_addr learned my addr 172.21.15.49:0/2784644891 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4c884640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48101810 0x7f1b48197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4d085640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 0x7f1b4819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 -- 172.21.15.49:0/2784644891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48101810 msgr2=0x7f1b48197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48101810 0x7f1b48197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 -- 172.21.15.49:0/2784644891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 msgr2=0x7f1b4819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 0x7f1b4819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 -- 172.21.15.49:0/2784644891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4819f6c0 con 0x7f1b48102a00 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b4d085640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 0x7f1b4819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b47fff640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9f0 tx=0x7f1b3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.700+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800cd80 con 0x7f1b48102a00 2024-03-16T08:16:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.701+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b38005500 con 0x7f1b48102a00 2024-03-16T08:16:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.701+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4819f9b0 con 0x7f1b48102a00 2024-03-16T08:16:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.701+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38010690 con 0x7f1b48102a00 2024-03-16T08:16:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.701+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4819ff00 con 0x7f1b48102a00 2024-03-16T08:16:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.702+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b48108ad0 con 0x7f1b48102a00 2024-03-16T08:16:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.703+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f1b380046c0 con 0x7f1b48102a00 2024-03-16T08:16:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.703+0000 7f1b45ffb640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 0x7f1b340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.704+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f1b380cefd0 con 0x7f1b48102a00 2024-03-16T08:16:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.704+0000 7f1b4c884640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 0x7f1b340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.704+0000 7f1b4c884640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 0x7f1b340a4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b48102860 tx=0x7f1b40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.708+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b3809bf40 con 0x7f1b48102a00 2024-03-16T08:16:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:27 smithi154 ceph-mon[36471]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:27 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1570699641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:27 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/527059563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:27 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1311635830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:27.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.937+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4802d1b0 con 0x7f1b48102a00 2024-03-16T08:16:27.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.938+0000 7f1b45ffb640 1 -- 172.21.15.49:0/2784644891 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f1b3809b840 con 0x7f1b48102a00 2024-03-16T08:16:27.938 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:27.938 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:16:27.938 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:27.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 msgr2=0x7f1b340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 0x7f1b340a4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b48102860 tx=0x7f1b40009500 comp rx=0 tx=0).stop 2024-03-16T08:16:27.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 msgr2=0x7f1b48197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:27.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9f0 tx=0x7f1b3800edf0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 shutdown_connections 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b481033c0 0x7f1b4819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1b340a2910 0x7f1b340a4dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102a00 0x7f1b48197fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 --2- 172.21.15.49:0/2784644891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b48101810 0x7f1b48197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 >> 172.21.15.49:0/2784644891 conn(0x7f1b480fcdc0 msgr2=0x7f1b48105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 shutdown_connections 2024-03-16T08:16:27.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:27.940+0000 7f1b4eae8640 1 -- 172.21.15.49:0/2784644891 wait complete. 2024-03-16T08:16:27.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 -- 172.21.15.49:0/4170484931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 msgr2=0x7fb418102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 --2- 172.21.15.49:0/4170484931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418102e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009860 tx=0x7fb40c02bb70 comp rx=0 tx=0).stop 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 -- 172.21.15.49:0/4170484931 shutdown_connections 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 --2- 172.21.15.49:0/4170484931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb418109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 --2- 172.21.15.49:0/4170484931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418102e80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 --2- 172.21.15.49:0/4170484931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 -- 172.21.15.49:0/4170484931 >> 172.21.15.49:0/4170484931 conn(0x7fb4180fcdc0 msgr2=0x7fb4180ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 -- 172.21.15.49:0/4170484931 shutdown_connections 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.060+0000 7fb41db48640 1 -- 172.21.15.49:0/4170484931 wait complete. 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 Processor -- start 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 -- start start 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb41819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d490 con 0x7fb418101810 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d310 con 0x7fb418102a00 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb41db48640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d610 con 0x7fb4181033c0 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb416ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb417fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb41819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36064/0 (socket says 172.21.15.49:36064) 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb416ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:57326/0 (socket says 172.21.15.49:57326) 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.061+0000 7fb4177fe640 1 -- 172.21.15.49:0/4097615191 learned_addr learned my addr 172.21.15.49:0/4097615191 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 -- 172.21.15.49:0/4097615191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 msgr2=0x7fb41819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb41819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 -- 172.21.15.49:0/4097615191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 msgr2=0x7fb418198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 -- 172.21.15.49:0/4097615191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819f6f0 con 0x7fb418101810 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb417fff640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb41819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb416ffd640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb4177fe640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9a0 tx=0x7fb40800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800cd80 con 0x7fb418101810 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819f9e0 con 0x7fb418101810 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408010910 con 0x7fb418101810 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408018a30 con 0x7fb418101810 2024-03-16T08:16:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.062+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41819ff00 con 0x7fb418101810 2024-03-16T08:16:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.063+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418108b30 con 0x7fb418101810 2024-03-16T08:16:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.064+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb408010ac0 con 0x7fb418101810 2024-03-16T08:16:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.065+0000 7fb414ff9640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 0x7fb4000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.065+0000 7fb416ffd640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 0x7fb4000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.065+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fb4080cef20 con 0x7fb418101810 2024-03-16T08:16:28.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.065+0000 7fb416ffd640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 0x7fb4000a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb40c02c180 tx=0x7fb40c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.069+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb408053740 con 0x7fb418101810 2024-03-16T08:16:28.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.254+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb41802d3c0 con 0x7fb418101810 2024-03-16T08:16:28.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.255+0000 7fb414ff9640 1 -- 172.21.15.49:0/4097615191 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fb40809be90 con 0x7fb418101810 2024-03-16T08:16:28.256 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:28.256 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.257+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 msgr2=0x7fb4000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.257+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 0x7fb4000a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb40c02c180 tx=0x7fb40c02e040 comp rx=0 tx=0).stop 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.257+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 msgr2=0x7fb418197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.257+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9a0 tx=0x7fb40800ee60 comp rx=0 tx=0).stop 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 shutdown_connections 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4181033c0 0x7fb41819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb418102a00 0x7fb418198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb4000a2890 0x7fb4000a4d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 --2- 172.21.15.49:0/4097615191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb418101810 0x7fb418197ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 >> 172.21.15.49:0/4097615191 conn(0x7fb4180fcdc0 msgr2=0x7fb418105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 shutdown_connections 2024-03-16T08:16:28.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.258+0000 7fb41db48640 1 -- 172.21.15.49:0/4097615191 wait complete. 2024-03-16T08:16:28.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- 172.21.15.49:0/483613927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 msgr2=0x7f7f48101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- 172.21.15.49:0/483613927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48101d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009900 tx=0x7f7f4402bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- 172.21.15.49:0/483613927 shutdown_connections 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- 172.21.15.49:0/483613927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f481022b0 0x7f7f48108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- 172.21.15.49:0/483613927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48101d70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- 172.21.15.49:0/483613927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f48100700 0x7f7f48100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- 172.21.15.49:0/483613927 >> 172.21.15.49:0/483613927 conn(0x7f7f480fbc70 msgr2=0x7f7f480fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- 172.21.15.49:0/483613927 shutdown_connections 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- 172.21.15.49:0/483613927 wait complete. 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 Processor -- start 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- start start 2024-03-16T08:16:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f481022b0 0x7f7f4819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d4a0 con 0x7f7f48100700 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d320 con 0x7f7f481022b0 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4f285640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d620 con 0x7f7f481018f0 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.378+0000 7f7f4d021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36084/0 (socket says 172.21.15.49:36084) 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 -- 172.21.15.49:0/657067928 learned_addr learned my addr 172.21.15.49:0/657067928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4c820640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d822640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f481022b0 0x7f7f4819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 -- 172.21.15.49:0/657067928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 msgr2=0x7f7f48198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 -- 172.21.15.49:0/657067928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f481022b0 msgr2=0x7f7f4819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f481022b0 0x7f7f4819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 -- 172.21.15.49:0/657067928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819f770 con 0x7f7f48100700 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4d021640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e9c0 tx=0x7f7f3800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800ccc0 con 0x7f7f48100700 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4819fa60 con 0x7f7f48100700 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f380035f0 con 0x7f7f48100700 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800ce60 con 0x7f7f48100700 2024-03-16T08:16:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.379+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4819ff80 con 0x7f7f48100700 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.380+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48107a70 con 0x7f7f48100700 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.381+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7f380037a0 con 0x7f7f48100700 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.382+0000 7f7f3e7fc640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 0x7f7f340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.382+0000 7f7f4c820640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 0x7f7f340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.382+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f7f380cef00 con 0x7f7f48100700 2024-03-16T08:16:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.382+0000 7f7f4c820640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 0x7f7f340a4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402c7b0 tx=0x7f7f44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.386+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f3809be70 con 0x7f7f48100700 2024-03-16T08:16:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:28 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2274486851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:28 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2784644891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:28 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4097615191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:28 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2274486851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:28 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2784644891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:28 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4097615191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.610+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4802d1b0 con 0x7f7f48100700 2024-03-16T08:16:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.611+0000 7f7f3e7fc640 1 -- 172.21.15.49:0/657067928 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f7f3809b770 con 0x7f7f48100700 2024-03-16T08:16:28.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:28.612 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:16:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:28.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 msgr2=0x7f7f340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 0x7f7f340a4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402c7b0 tx=0x7f7f44005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 msgr2=0x7f7f48197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e9c0 tx=0x7f7f3800edc0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 shutdown_connections 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f481022b0 0x7f7f4819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f481018f0 0x7f7f48198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f7f340a2890 0x7f7f340a4d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 --2- 172.21.15.49:0/657067928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f48100700 0x7f7f48197b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 >> 172.21.15.49:0/657067928 conn(0x7f7f480fbc70 msgr2=0x7f7f48104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 shutdown_connections 2024-03-16T08:16:28.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.613+0000 7f7f4f285640 1 -- 172.21.15.49:0/657067928 wait complete. 2024-03-16T08:16:28.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T08:15:01.568993+0000', 'modified': '2024-03-16T08:15:07.570394+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24505, 'mds_2': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/272945510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 272945510}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 272945510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/2047612746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 2047612746}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 2047612746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/4234449434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 4234449434}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4234449434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.129:6837/1811728297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 1811728297}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 1811728297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.154:6839/1693946357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 1693946357}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 1693946357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6839/1288779286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 1288779286}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 1288779286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24536, 24491, 24505]} 2024-03-16T08:16:28.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-16T08:16:28.625 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T08:16:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:28 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2274486851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:28 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2784644891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:28 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4097615191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 -- 172.21.15.49:0/1596284271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b81006e0 msgr2=0x7ff5b8100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 --2- 172.21.15.49:0/1596284271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b81006e0 0x7ff5b8100b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009900 tx=0x7ff5b002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 -- 172.21.15.49:0/1596284271 shutdown_connections 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 --2- 172.21.15.49:0/1596284271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b8102290 0x7ff5b8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 --2- 172.21.15.49:0/1596284271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81018d0 0x7ff5b8101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 --2- 172.21.15.49:0/1596284271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b81006e0 0x7ff5b8100b00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 -- 172.21.15.49:0/1596284271 >> 172.21.15.49:0/1596284271 conn(0x7ff5b80fbc70 msgr2=0x7ff5b80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 -- 172.21.15.49:0/1596284271 shutdown_connections 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.733+0000 7ff5bd148640 1 -- 172.21.15.49:0/1596284271 wait complete. 2024-03-16T08:16:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 Processor -- start 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 -- start start 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81006e0 0x7ff5b8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b81018d0 0x7ff5b81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b810d480 con 0x7ff5b8102290 2024-03-16T08:16:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b810d300 con 0x7ff5b81018d0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5bd148640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b810d600 con 0x7ff5b81006e0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.734+0000 7ff5b759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81006e0 0x7ff5b8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36110/0 (socket says 172.21.15.49:36110) 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 -- 172.21.15.49:0/3267036204 learned_addr learned my addr 172.21.15.49:0/3267036204 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b659c640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b81018d0 0x7ff5b81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 -- 172.21.15.49:0/3267036204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81006e0 msgr2=0x7ff5b8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81006e0 0x7ff5b8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 -- 172.21.15.49:0/3267036204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b81018d0 msgr2=0x7ff5b81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b81018d0 0x7ff5b81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 -- 172.21.15.49:0/3267036204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b819f7a0 con 0x7ff5b8102290 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5b759e640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb10 tx=0x7ff5a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff59ffff640 1 -- 172.21.15.49:0/3267036204 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004b20 con 0x7ff5b8102290 2024-03-16T08:16:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b819fa90 con 0x7ff5b8102290 2024-03-16T08:16:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff59ffff640 1 -- 172.21.15.49:0/3267036204 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a8004cc0 con 0x7ff5b8102290 2024-03-16T08:16:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff59ffff640 1 -- 172.21.15.49:0/3267036204 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010760 con 0x7ff5b8102290 2024-03-16T08:16:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.735+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b819ffe0 con 0x7ff5b8102290 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.737+0000 7ff59ffff640 1 -- 172.21.15.49:0/3267036204 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff5a8005020 con 0x7ff5b8102290 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 Processor -- start 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 -- start start 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff59ffff640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 0x7ff5a00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c46d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c46d730 0x55884c46fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55884c3cbc80 con 0x55884c4685c0 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55884c374aa0 con 0x55884c46d730 2024-03-16T08:16:28.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55884c3c7d20 con 0x55884c467bd0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b659c640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 0x7ff5a00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b759e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c46d730 0x55884c46fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:35690/0 (socket says 172.21.15.49:35690) 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/2721507561 learned_addr learned my addr 172.21.15.49:0/2721507561 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff5b659c640 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c46d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.738+0000 7ff59ffff640 1 -- 172.21.15.49:0/3267036204 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff5a80cf130 con 0x7ff5b8102290 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/2721507561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c46d730 msgr2=0x55884c46fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c46d730 0x55884c46fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/2721507561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 msgr2=0x55884c46d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c46d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/2721507561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884c396ff0 con 0x55884c467bd0 2024-03-16T08:16:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b659c640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 0x7ff5a00a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009fd0 tx=0x7ff5ac009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009900 tx=0x7ff5b0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16edd8f7d6c7f8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff59d7fa640 1 -- 172.21.15.49:0/2721507561 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b002cb80 con 0x55884c467bd0 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff59d7fa640 1 -- 172.21.15.49:0/2721507561 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0005040 con 0x55884c467bd0 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.739+0000 7ff59d7fa640 1 -- 172.21.15.49:0/2721507561 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0004920 con 0x55884c467bd0 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2721507561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 msgr2=0x55884c467ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009900 tx=0x7ff5b0009560 comp rx=0 tx=0).stop 2024-03-16T08:16:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2721507561 shutdown_connections 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c46d730 0x55884c46fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c46d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2721507561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c467ff0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2721507561 >> 172.21.15.49:0/2721507561 conn(0x55884c463140 msgr2=0x55884c463540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2721507561 shutdown_connections 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2721507561 wait complete. 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- start start 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 Processor -- start 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 0x55884c471d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c479300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55884c470680 con 0x55884c4685c0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55884c387ef0 con 0x55884c467bd0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55884c470800 con 0x55884c46d730 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5bfe0b740 10 client.0 _ll_register_callbacks cb 0x55884c382dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.740+0000 7ff5b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36130/0 (socket says 172.21.15.49:36130) 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 0x55884c471d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 -- 172.21.15.49:0/2472922370 learned_addr learned my addr 172.21.15.49:0/2472922370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b759e640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c479300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 -- 172.21.15.49:0/2472922370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 msgr2=0x55884c479300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c479300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 -- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 msgr2=0x55884c471d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 0x55884c471d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 -- 172.21.15.49:0/2472922370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884c394a30 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 0x55884c471d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5b659c640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009f60 tx=0x7ff5ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b52729ad67b31f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff59aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0044c0 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b0005040 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff5ac003800 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff5ac003a00 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5bfe0b740 10 client.15048 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55884c0e02d0 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.741+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0005040 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff5ac01fbb0 con 0x55884c4685c0 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 10 client.15048 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 20 client.15048 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c47e930 0x55884c47fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 4 client.15048 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55884c3d91c0 con 0x55884c47e930 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c47e930 0x55884c47fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff59aff5640 10 client.15048 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.742+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c47e930 0x55884c47fda0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d6d60efba3997af server_cookie=c4dc3b2b19f375d9 in_seq=0 out_seq=0 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff59aff5640 1 -- 172.21.15.49:0/2472922370 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55884c3d91c0 con 0x55884c47e930 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff59aff5640 10 client.15048 handle_command_reply: tid=0 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 client.15048 shutdown 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 msgr2=0x55884c4722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009f60 tx=0x7ff5ac01d000 comp rx=0 tx=0).stop 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 shutdown_connections 2024-03-16T08:16:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c47e930 0x55884c47fda0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c479300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c4685c0 0x55884c4722c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff59aff5640 0 client.15048 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/2472922370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c467bd0 0x55884c471d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 >> 172.21.15.49:0/2472922370 conn(0x55884c463140 msgr2=0x55884c462380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 shutdown_connections 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/2472922370 wait complete. 2024-03-16T08:16:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.743+0000 7ff5bfe0b740 20 client.15048 trim_cache size 0 max 16384 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 Processor -- start 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 -- start start 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c4792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c471ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55884c47e0e0 con 0x55884c467bd0 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55884c47e7b0 con 0x55884c4685c0 2024-03-16T08:16:28.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.761+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55884c47ee80 con 0x55884c46d730 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36136/0 (socket says 172.21.15.49:36136) 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/3989848347 learned_addr learned my addr 172.21.15.49:0/3989848347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b759e640 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c471ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b659c640 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c4792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/3989848347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 msgr2=0x55884c471ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c471ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/3989848347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 msgr2=0x55884c4792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c4792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 -- 172.21.15.49:0/3989848347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884c1aa6b0 con 0x55884c467bd0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55884c468e20 tx=0x7ff5b002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3852cfc6a97135 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff59cff9640 1 -- 172.21.15.49:0/3989848347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00054f0 con 0x55884c467bd0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff59cff9640 1 -- 172.21.15.49:0/3989848347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b00510d0 con 0x55884c467bd0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.762+0000 7ff59cff9640 1 -- 172.21.15.49:0/3989848347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0003850 con 0x55884c467bd0 2024-03-16T08:16:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/3989848347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 msgr2=0x55884c471990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55884c468e20 tx=0x7ff5b002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/3989848347 shutdown_connections 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c46d730 0x55884c471ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c4792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/3989848347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c467bd0 0x55884c471990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/3989848347 >> 172.21.15.49:0/3989848347 conn(0x55884c463140 msgr2=0x55884c46cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/3989848347 shutdown_connections 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/3989848347 wait complete. 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- start start 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 Processor -- start 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c4741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c474730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55884c47fc20 con 0x55884c46d730 2024-03-16T08:16:28.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55884c47f550 con 0x55884c4685c0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55884c4802f0 con 0x55884c467bd0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.763+0000 7ff5bfe0b740 10 client.0 _ll_register_callbacks cb 0x7ff5ac009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36140/0 (socket says 172.21.15.49:36140) 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 -- 172.21.15.49:0/977144750 learned_addr learned my addr 172.21.15.49:0/977144750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b659c640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c474730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c4741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 -- 172.21.15.49:0/977144750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 msgr2=0x55884c4741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c4741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 -- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 msgr2=0x55884c474730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c474730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 -- 172.21.15.49:0/977144750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884c4e2bd0 con 0x55884c46d730 2024-03-16T08:16:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5b759e640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005880 tx=0x7ff5a809b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1c6dfaaa81bab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff59aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ae170 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff5a80c5740 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b00510d0 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff5a80c5960 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5bfe0b740 10 client.15054 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55884c4e3eb0 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.764+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b00510d0 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff5a8005310 con 0x55884c46d730 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 10 client.15054 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 20 client.15054 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c4e8a80 0x55884c4e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 4 client.15054 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "status"}) -- 0x55884c3d3510 con 0x55884c4e8a80 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c4e8a80 0x55884c4e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff59aff5640 10 client.15054 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5b6d9d640 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c4e8a80 0x55884c4e7790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f23adccc2418c72 server_cookie=3a8214c5fa3c128c in_seq=0 out_seq=0 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff59aff5640 1 -- 172.21.15.49:0/977144750 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55884c3d3510 con 0x55884c4e8a80 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff59aff5640 10 client.15054 handle_command_reply: tid=0 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 1 client.15054 shutdown 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 msgr2=0x55884c4e2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.765+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005880 tx=0x7ff5a809b3c0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 shutdown_connections 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55884c4e8a80 0x55884c4e7790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55884c46d730 0x55884c4e2690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55884c4685c0 0x55884c474730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff59aff5640 0 client.15054 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 --2- 172.21.15.49:0/977144750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884c467bd0 0x55884c4741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 >> 172.21.15.49:0/977144750 conn(0x55884c463140 msgr2=0x55884c471990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 shutdown_connections 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 1 -- 172.21.15.49:0/977144750 wait complete. 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bfe0b740 20 client.15054 trim_cache size 0 max 16384 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24491, 2024-03-16T08:16:28.773 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 82.200669761, 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 102.879774799 2024-03-16T08:16:28.774 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 msgr2=0x7ff5a00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 0x7ff5a00a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009fd0 tx=0x7ff5ac009b80 comp rx=0 tx=0).stop 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 msgr2=0x7ff5b819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb10 tx=0x7ff5a800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 shutdown_connections 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff5a00a2890 0x7ff5a00a4d50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b8102290 0x7ff5b819f130 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5b81018d0 0x7ff5b81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 --2- 172.21.15.49:0/3267036204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5b81006e0 0x7ff5b8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 >> 172.21.15.49:0/3267036204 conn(0x7ff5b80fbc70 msgr2=0x7ff5b8104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 shutdown_connections 2024-03-16T08:16:28.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.766+0000 7ff5bd148640 1 -- 172.21.15.49:0/3267036204 wait complete. 2024-03-16T08:16:28.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 -- 172.21.15.49:0/4085353199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73281033c0 msgr2=0x7f7328109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 --2- 172.21.15.49:0/4085353199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73281033c0 0x7f7328109c50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7320009ca0 tx=0x7f732002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 -- 172.21.15.49:0/4085353199 shutdown_connections 2024-03-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 --2- 172.21.15.49:0/4085353199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73281033c0 0x7f7328109c50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 --2- 172.21.15.49:0/4085353199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7328102a00 0x7f7328102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 --2- 172.21.15.49:0/4085353199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 0x7f7328101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 -- 172.21.15.49:0/4085353199 >> 172.21.15.49:0/4085353199 conn(0x7f73280fcd80 msgr2=0x7f73280ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 -- 172.21.15.49:0/4085353199 shutdown_connections 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.886+0000 7f732e412640 1 -- 172.21.15.49:0/4085353199 wait complete. 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 Processor -- start 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 -- start start 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 0x7f7328197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73281033c0 0x7f732819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f732810d580 con 0x7f7328102a00 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f732810d400 con 0x7f7328101810 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732e412640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f732810d700 con 0x7f73281033c0 2024-03-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36150/0 (socket says 172.21.15.49:36150) 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 -- 172.21.15.49:0/184460897 learned_addr learned my addr 172.21.15.49:0/184460897 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f732c9af640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73281033c0 0x7f732819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f7327fff640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 0x7f7328197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 -- 172.21.15.49:0/184460897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73281033c0 msgr2=0x7f732819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73281033c0 0x7f732819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 -- 172.21.15.49:0/184460897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 msgr2=0x7f7328197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 0x7f7328197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.887+0000 7f73277fe640 1 -- 172.21.15.49:0/184460897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732819f7c0 con 0x7f7328102a00 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f73277fe640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f731c00b4a0 tx=0x7f731c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f73257fa640 1 -- 172.21.15.49:0/184460897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c015020 con 0x7f7328102a00 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732819fab0 con 0x7f7328102a00 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f73257fa640 1 -- 172.21.15.49:0/184460897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c011d10 con 0x7f7328102a00 2024-03-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f73257fa640 1 -- 172.21.15.49:0/184460897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c00f2e0 con 0x7f7328102a00 2024-03-16T08:16:28.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.888+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73281a0050 con 0x7f7328102a00 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f73257fa640 1 -- 172.21.15.49:0/184460897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f731c018020 con 0x7f7328102a00 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 Processor -- start 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 -- start start 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 0x56159f0830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f085bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56159efe1c80 con 0x56159f07dbd0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56159ef8aaa0 con 0x56159f083730 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f7331001740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56159efddd20 con 0x56159f07e5c0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f732c9af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f732c9af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36156/0 (socket says 172.21.15.49:36156) 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.890+0000 7f732c9af640 1 -- 172.21.15.49:0/2432110105 learned_addr learned my addr 172.21.15.49:0/2432110105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f73257fa640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 0x7f73100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 0x56159f0830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f73277fe640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f085bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f732c9af640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 0x7f73100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 -- 172.21.15.49:0/2432110105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 msgr2=0x56159f085bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f085bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 -- 172.21.15.49:0/2432110105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 msgr2=0x56159f07dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 -- 172.21.15.49:0/2432110105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159efacff0 con 0x56159f07e5c0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f73277fe640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f085bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f732c9af640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f73257fa640 1 -- 172.21.15.49:0/184460897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f731c0ce300 con 0x7f7328102a00 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f7327fff640 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 0x56159f0830c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7318009970 tx=0x7f7318009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada6ab2e7bc90641 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f732c9af640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 0x7f73100a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7320032420 tx=0x7f732002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f730e7fc640 1 -- 172.21.15.49:0/2432110105 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318017070 con 0x56159f07e5c0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f730e7fc640 1 -- 172.21.15.49:0/2432110105 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7318013470 con 0x56159f07e5c0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.891+0000 7f730e7fc640 1 -- 172.21.15.49:0/2432110105 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318012630 con 0x56159f07e5c0 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- 172.21.15.49:0/2432110105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 msgr2=0x56159f0830c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 0x56159f0830c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7318009970 tx=0x7f7318009e30 comp rx=0 tx=0).stop 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- 172.21.15.49:0/2432110105 shutdown_connections 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f085bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07e5c0 0x56159f0830c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 --2- 172.21.15.49:0/2432110105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07dbd0 0x56159f07dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- 172.21.15.49:0/2432110105 >> 172.21.15.49:0/2432110105 conn(0x56159f079140 msgr2=0x56159f079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- 172.21.15.49:0/2432110105 shutdown_connections 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- 172.21.15.49:0/2432110105 wait complete. 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 -- start start 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.892+0000 7f7331001740 1 Processor -- start 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f088330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f08f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56159f0866e0 con 0x56159f07e5c0 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56159ef9def0 con 0x56159f083730 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56159f086860 con 0x56159f07dbd0 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7331001740 10 client.0 _ll_register_callbacks cb 0x56159ef98dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f088330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:35738/0 (socket says 172.21.15.49:35738) 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 -- 172.21.15.49:0/409444710 learned_addr learned my addr 172.21.15.49:0/409444710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f732c9af640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f08f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 -- 172.21.15.49:0/409444710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 msgr2=0x56159f08f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f08f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 -- 172.21.15.49:0/409444710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 msgr2=0x56159f088330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f088330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 -- 172.21.15.49:0/409444710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159efaaa30 con 0x56159f07dbd0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f73277fe640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f088330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f7327fff640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73180092f0 tx=0x7f731800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87338281f0ae95ce server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.893+0000 7f73077fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318017050 con 0x56159f07dbd0 2024-03-16T08:16:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f731801c450 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7318013470 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f731802d070 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 10 client.24949 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56159ecf62d0 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7318013470 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7318026740 con 0x56159f07dbd0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 10 client.24949 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 10 client.24949 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 20 client.24949 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0943d0 0x56159f095840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 4 client.24949 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56159efef1c0 con 0x56159f0943d0 2024-03-16T08:16:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.894+0000 7f73277fe640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0943d0 0x56159f095840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.895+0000 7f73077fe640 10 client.24949 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.895+0000 7f73277fe640 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0943d0 0x56159f095840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20c784179244da37 server_cookie=708991446d1bc6fb in_seq=0 out_seq=0 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.895+0000 7f73077fe640 1 -- 172.21.15.49:0/409444710 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56159efef1c0 con 0x56159f0943d0 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.895+0000 7f73077fe640 10 client.24949 handle_command_reply: tid=0 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.895+0000 7f7331001740 1 client.24949 shutdown 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 msgr2=0x56159f087df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73180092f0 tx=0x7f731800ef60 comp rx=0 tx=0).stop 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 shutdown_connections 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0943d0 0x56159f095840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f08f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f088330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f73077fe640 0 client.24949 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 --2- 172.21.15.49:0/409444710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f087df0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 >> 172.21.15.49:0/409444710 conn(0x56159f079140 msgr2=0x56159f078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 shutdown_connections 2024-03-16T08:16:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 1 -- 172.21.15.49:0/409444710 wait complete. 2024-03-16T08:16:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.896+0000 7f7331001740 20 client.24949 trim_cache size 0 max 16384 2024-03-16T08:16:28.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 Processor -- start 2024-03-16T08:16:28.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 -- start start 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f085b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f079520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56159f093b80 con 0x56159f07e5c0 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56159f094250 con 0x56159f083730 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7331001740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56159f094920 con 0x56159f07dbd0 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36162/0 (socket says 172.21.15.49:36162) 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f73277fe640 1 -- 172.21.15.49:0/2911752881 learned_addr learned my addr 172.21.15.49:0/2911752881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f7327fff640 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f085b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.914+0000 7f732c9af640 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f079520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 -- 172.21.15.49:0/2911752881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 msgr2=0x56159f085b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f085b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 -- 172.21.15.49:0/2911752881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 msgr2=0x56159f079520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f079520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 -- 172.21.15.49:0/2911752881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159edc06b0 con 0x56159f07e5c0 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f73277fe640 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f731c0970e0 tx=0x7f731c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c766638d2cad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f730dffb640 1 -- 172.21.15.49:0/2911752881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c09edd0 con 0x56159f07e5c0 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f730dffb640 1 -- 172.21.15.49:0/2911752881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c0a3420 con 0x56159f07e5c0 2024-03-16T08:16:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.915+0000 7f730dffb640 1 -- 172.21.15.49:0/2911752881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c09a0d0 con 0x56159f07e5c0 2024-03-16T08:16:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- 172.21.15.49:0/2911752881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 msgr2=0x56159f08f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f731c0970e0 tx=0x7f731c0047d0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- 172.21.15.49:0/2911752881 shutdown_connections 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f079520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f08f330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 --2- 172.21.15.49:0/2911752881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f085b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- 172.21.15.49:0/2911752881 >> 172.21.15.49:0/2911752881 conn(0x56159f079140 msgr2=0x56159f07cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- 172.21.15.49:0/2911752881 shutdown_connections 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- 172.21.15.49:0/2911752881 wait complete. 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 -- start start 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.916+0000 7f7331001740 1 Processor -- start 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f081370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f0f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56159f0956c0 con 0x56159f07e5c0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56159f094ff0 con 0x56159f083730 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56159f095d90 con 0x56159f07dbd0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 10 client.0 _ll_register_callbacks cb 0x7f7320009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36176/0 (socket says 172.21.15.49:36176) 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 -- 172.21.15.49:0/2242161623 learned_addr learned my addr 172.21.15.49:0/2242161623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7327fff640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f081370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f732c9af640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f0f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 -- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 msgr2=0x56159f081370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f081370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 -- 172.21.15.49:0/2242161623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 msgr2=0x56159f0f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f0f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 -- 172.21.15.49:0/2242161623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159f08f420 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73277fe640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f731c09ac10 tx=0x7f731c096f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3194afa56addf18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73077fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0b3070 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f731c0972f0 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.917+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c0a3420 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f731c09a0d0 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 10 client.15072 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56159f0f94f0 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c0a3420 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f731c0cd040 con 0x56159f07e5c0 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 10 client.15072 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 20 client.15072 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0fdaf0 0x56159f0fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 4 client.15072 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "status"}) -- 0x56159efe9510 con 0x56159f0fdaf0 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.918+0000 7f7327fff640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0fdaf0 0x56159f0fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f73077fe640 10 client.15072 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7327fff640 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0fdaf0 0x56159f0fa600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94e7593eed8d6d7e server_cookie=210182c9505d4e41 in_seq=0 out_seq=0 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f73077fe640 1 -- 172.21.15.49:0/2242161623 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56159efe9510 con 0x56159f0fdaf0 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f73077fe640 10 client.15072 handle_command_reply: tid=0 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 client.15072 shutdown 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 msgr2=0x56159f0818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f731c09ac10 tx=0x7f731c096f20 comp rx=0 tx=0).stop 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 shutdown_connections 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56159f0fdaf0 0x56159f0fa600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56159f083730 0x56159f0f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56159f07e5c0 0x56159f0818b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f73077fe640 0 client.15072 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 --2- 172.21.15.49:0/2242161623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56159f07dbd0 0x56159f081370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 >> 172.21.15.49:0/2242161623 conn(0x56159f079140 msgr2=0x56159f07cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 shutdown_connections 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 1 -- 172.21.15.49:0/2242161623 wait complete. 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.919+0000 7f7331001740 20 client.15072 trim_cache size 0 max 16384 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-16T08:16:28.926 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24505, 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 83.362497626999996, 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 100.293303722 2024-03-16T08:16:28.927 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 msgr2=0x7f73100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 0x7f73100a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7320032420 tx=0x7f732002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 msgr2=0x7f73281980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f731c00b4a0 tx=0x7f731c00b8a0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 shutdown_connections 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73281033c0 0x7f732819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f73100a2890 0x7f73100a4d50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7328102a00 0x7f73281980e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 --2- 172.21.15.49:0/184460897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7328101810 0x7f7328197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 >> 172.21.15.49:0/184460897 conn(0x7f73280fcd80 msgr2=0x7f7328105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 shutdown_connections 2024-03-16T08:16:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:28.920+0000 7f732e412640 1 -- 172.21.15.49:0/184460897 wait complete. 2024-03-16T08:16:28.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T08:16:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- 172.21.15.49:0/4031019477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a4100230 msgr2=0x7fb1a4109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/4031019477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a4100230 0x7fb1a4109de0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009ca0 tx=0x7fb19c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- 172.21.15.49:0/4031019477 shutdown_connections 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/4031019477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a4100230 0x7fb1a4109de0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/4031019477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a40ff870 0x7fb1a40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/4031019477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 0x7fb1a4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- 172.21.15.49:0/4031019477 >> 172.21.15.49:0/4031019477 conn(0x7fb1a40fb410 msgr2=0x7fb1a40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- 172.21.15.49:0/4031019477 shutdown_connections 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- 172.21.15.49:0/4031019477 wait complete. 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 Processor -- start 2024-03-16T08:16:29.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.040+0000 7fb1aa86c640 1 -- start start 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a4100230 0x7fb1a4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 0x7fb1a419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d480 con 0x7fb1a40ff870 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d300 con 0x7fb1a4100230 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d600 con 0x7fb1a4102e00 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36198/0 (socket says 172.21.15.49:36198) 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 -- 172.21.15.49:0/3539475588 learned_addr learned my addr 172.21.15.49:0/3539475588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a4100230 0x7fb1a4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 0x7fb1a419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 -- 172.21.15.49:0/3539475588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 msgr2=0x7fb1a419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 0x7fb1a419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 -- 172.21.15.49:0/3539475588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a4100230 msgr2=0x7fb1a4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a4100230 0x7fb1a4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 -- 172.21.15.49:0/3539475588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a419f6c0 con 0x7fb1a40ff870 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb19400e9c0 tx=0x7fb19400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a17fa640 1 -- 172.21.15.49:0/3539475588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19400cd80 con 0x7fb1a40ff870 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a419f9b0 con 0x7fb1a40ff870 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a17fa640 1 -- 172.21.15.49:0/3539475588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1940035f0 con 0x7fb1a40ff870 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.041+0000 7fb1a17fa640 1 -- 172.21.15.49:0/3539475588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194010690 con 0x7fb1a40ff870 2024-03-16T08:16:29.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.042+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a419ff00 con 0x7fb1a40ff870 2024-03-16T08:16:29.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a17fa640 1 -- 172.21.15.49:0/3539475588 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb1940037a0 con 0x7fb1a40ff870 2024-03-16T08:16:29.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 Processor -- start 2024-03-16T08:16:29.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 -- start start 2024-03-16T08:16:29.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b80dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b8130c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b50b771c80 con 0x55b50b813730 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50b71aaa0 con 0x55b50b80dbd0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b50b76dd20 con 0x55b50b80e5c0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a17fa640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 0x7fb18c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b8130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36200/0 (socket says 172.21.15.49:36200) 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 -- 172.21.15.49:0/967493999 learned_addr learned my addr 172.21.15.49:0/967493999 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b80dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 0x7fb18c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a17fa640 1 -- 172.21.15.49:0/3539475588 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fb1940cef20 con 0x7fb1a40ff870 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 -- 172.21.15.49:0/967493999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 msgr2=0x55b50b8130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b8130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 -- 172.21.15.49:0/967493999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 msgr2=0x55b50b80dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.044+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b80dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb1a37fe640 1 -- 172.21.15.49:0/967493999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50b73cff0 con 0x55b50b813730 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 0x7fb18c0a4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005e30 tx=0x7fb19c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb198009c90 tx=0x7fb198011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29965c8176cdafe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb18a7fc640 1 -- 172.21.15.49:0/967493999 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19801e070 con 0x55b50b813730 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb18a7fc640 1 -- 172.21.15.49:0/967493999 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb198019b10 con 0x55b50b813730 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb18a7fc640 1 -- 172.21.15.49:0/967493999 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19800e9c0 con 0x55b50b813730 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb1ad44c740 1 -- 172.21.15.49:0/967493999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 msgr2=0x55b50b815bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.045+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb198009c90 tx=0x7fb198011810 comp rx=0 tx=0).stop 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- 172.21.15.49:0/967493999 shutdown_connections 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b813730 0x55b50b815bc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b8130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/967493999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b80dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- 172.21.15.49:0/967493999 >> 172.21.15.49:0/967493999 conn(0x55b50b809140 msgr2=0x55b50b809540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- 172.21.15.49:0/967493999 shutdown_connections 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- 172.21.15.49:0/967493999 wait complete. 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- start start 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 Processor -- start 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 0x55b50b818260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b81f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b50b8166e0 con 0x55b50b80dbd0 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50b72def0 con 0x55b50b80e5c0 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b50b816860 con 0x55b50b813730 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.046+0000 7fb1ad44c740 10 client.0 _ll_register_callbacks cb 0x55b50b728dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:29.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 0x55b50b818260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36214/0 (socket says 172.21.15.49:36214) 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 -- 172.21.15.49:0/553367127 learned_addr learned my addr 172.21.15.49:0/553367127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b81f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 -- 172.21.15.49:0/553367127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 msgr2=0x55b50b81f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b81f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 -- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 msgr2=0x55b50b818260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 0x55b50b818260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 -- 172.21.15.49:0/553367127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50b73aa30 con 0x55b50b80dbd0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 0x55b50b818260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb19409b490 tx=0x7fb19409b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb93d9bce1614f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1837fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19409b790 con 0x55b50b80dbd0 2024-03-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fb1940860b0 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb198019b10 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb19409f8e0 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1ad44c740 10 client.15084 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b50b4862d0 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.047+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb198019b10 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb1940afb70 con 0x55b50b80dbd0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 20 client.15084 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b824820 0x55b50b825c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 4 client.15084 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50b77f1c0 con 0x55b50b824820 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b824820 0x55b50b825c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b824820 0x55b50b825c90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8068505eff4a27b8 server_cookie=d574f7688e6d6926 in_seq=0 out_seq=0 2024-03-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.048+0000 7fb1837fe640 10 client.15084 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.049+0000 7fb1837fe640 1 -- 172.21.15.49:0/553367127 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b50b77f1c0 con 0x55b50b824820 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.049+0000 7fb1837fe640 10 client.15084 handle_command_reply: tid=0 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.049+0000 7fb1ad44c740 1 client.15084 shutdown 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 msgr2=0x55b50b817d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb19409b490 tx=0x7fb19409b620 comp rx=0 tx=0).stop 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 shutdown_connections 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b824820 0x55b50b825c90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b81f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1837fe640 0 client.15084 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80e5c0 0x55b50b818260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/553367127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 >> 172.21.15.49:0/553367127 conn(0x55b50b809140 msgr2=0x55b50b808380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 shutdown_connections 2024-03-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 1 -- 172.21.15.49:0/553367127 wait complete. 2024-03-16T08:16:29.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.050+0000 7fb1ad44c740 20 client.15084 trim_cache size 0 max 16384 2024-03-16T08:16:29.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 Processor -- start 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 -- start start 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b81f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b813730 0x55b50b817e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b50b823fd0 con 0x55b50b80dbd0 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50b8246a0 con 0x55b50b813730 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b50b824d70 con 0x55b50b80e5c0 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.068+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36224/0 (socket says 172.21.15.49:36224) 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 -- 172.21.15.49:0/1573937265 learned_addr learned my addr 172.21.15.49:0/1573937265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:29.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b81f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b813730 0x55b50b817e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 -- 172.21.15.49:0/1573937265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 msgr2=0x55b50b81f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b81f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 -- 172.21.15.49:0/1573937265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b813730 msgr2=0x55b50b817e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b813730 0x55b50b817e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 -- 172.21.15.49:0/1573937265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50b5506b0 con 0x55b50b80dbd0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb1940058c0 tx=0x7fb194003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251589803d96e856 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb189ffb640 1 -- 172.21.15.49:0/1573937265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19409b0f0 con 0x55b50b80dbd0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb189ffb640 1 -- 172.21.15.49:0/1573937265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19409b790 con 0x55b50b80dbd0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.069+0000 7fb189ffb640 1 -- 172.21.15.49:0/1573937265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940a3aa0 con 0x55b50b80dbd0 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1573937265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 msgr2=0x55b50b817930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb1940058c0 tx=0x7fb194003f10 comp rx=0 tx=0).stop 2024-03-16T08:16:29.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1573937265 shutdown_connections 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b813730 0x55b50b817e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b80e5c0 0x55b50b81f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1573937265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80dbd0 0x55b50b817930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1573937265 >> 172.21.15.49:0/1573937265 conn(0x55b50b809140 msgr2=0x55b50b823640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1573937265 shutdown_connections 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1573937265 wait complete. 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- start start 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 Processor -- start 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b81d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b888ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b50b825b10 con 0x55b50b80e5c0 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50b825440 con 0x55b50b80dbd0 2024-03-16T08:16:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b50b8261e0 con 0x55b50b813730 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.070+0000 7fb1ad44c740 10 client.0 _ll_register_callbacks cb 0x7fb1940973c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b81d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36236/0 (socket says 172.21.15.49:36236) 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 -- 172.21.15.49:0/1458229822 learned_addr learned my addr 172.21.15.49:0/1458229822 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a8e09640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b888ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 -- 172.21.15.49:0/1458229822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 msgr2=0x55b50b888ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b888ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 -- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 msgr2=0x55b50b81d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b81d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 -- 172.21.15.49:0/1458229822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50b889000 con 0x55b50b80e5c0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b81d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1a37fe640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb198009be0 tx=0x7fb198019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc8aab48b5fcdc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1837fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19801e040 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fb198005110 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19409b790 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb198005350 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1ad44c740 10 client.15090 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b50b88a1d0 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.071+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19409b790 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb1980035f0 con 0x55b50b80e5c0 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 20 client.15090 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b88eba0 0x55b50b88d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 4 client.15090 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "status"}) -- 0x55b50b779510 con 0x55b50b88eba0 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b88eba0 0x55b50b88d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1837fe640 10 client.15090 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1a3fff640 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b88eba0 0x55b50b88d8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0ff93c332a7e0d server_cookie=a5f378558ef740a1 in_seq=0 out_seq=0 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1837fe640 1 -- 172.21.15.49:0/1458229822 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b50b779510 con 0x55b50b88eba0 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1837fe640 10 client.15090 handle_command_reply: tid=0 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.072+0000 7fb1ad44c740 1 client.15090 shutdown 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 msgr2=0x55b50b886570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb198009be0 tx=0x7fb198019740 comp rx=0 tx=0).stop 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 shutdown_connections 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b50b88eba0 0x55b50b88d8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b50b813730 0x55b50b888ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1837fe640 0 client.15090 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b50b80e5c0 0x55b50b886570 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 --2- 172.21.15.49:0/1458229822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50b80dbd0 0x55b50b81d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 >> 172.21.15.49:0/1458229822 conn(0x55b50b809140 msgr2=0x55b50b817910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 shutdown_connections 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 1 -- 172.21.15.49:0/1458229822 wait complete. 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1ad44c740 20 client.15090 trim_cache size 0 max 16384 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24536, 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:29.080 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 86.540153251999996, 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 92.065663127999997 2024-03-16T08:16:29.081 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 msgr2=0x7fb18c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 0x7fb18c0a4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005e30 tx=0x7fb19c032590 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 msgr2=0x7fb1a4197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb19400e9c0 tx=0x7fb19400ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 shutdown_connections 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a4102e00 0x7fb1a419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1a4100230 0x7fb1a4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb18c0a2890 0x7fb18c0a4d50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 --2- 172.21.15.49:0/3539475588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1a40ff870 0x7fb1a4197aa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 >> 172.21.15.49:0/3539475588 conn(0x7fb1a40fb410 msgr2=0x7fb1a4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 shutdown_connections 2024-03-16T08:16:29.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:29.073+0000 7fb1aa86c640 1 -- 172.21.15.49:0/3539475588 wait complete. 2024-03-16T08:16:29.084 INFO:tasks.cephadm:Setup complete, yielding 2024-03-16T08:16:29.085 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:16:29.095 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:16:29.095 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-16T08:16:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:29.602 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:29 smithi049 ceph-mon[31942]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:29.602 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:29 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/657067928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:29.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:29 smithi129 ceph-mon[36561]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:29.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:29 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/657067928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:29 smithi154 ceph-mon[36471]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:29.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:29 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/657067928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:30.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 -- 172.21.15.49:0/2802417587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701003d0 msgr2=0x7f6170100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 --2- 172.21.15.49:0/2802417587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701003d0 0x7f6170100850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6164009990 tx=0x7f6164009ca0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 -- 172.21.15.49:0/2802417587 shutdown_connections 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 --2- 172.21.15.49:0/2802417587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170100d90 0x7f6170101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 --2- 172.21.15.49:0/2802417587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61701028d0 0x7f6170102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 --2- 172.21.15.49:0/2802417587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701003d0 0x7f6170100850 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 -- 172.21.15.49:0/2802417587 >> 172.21.15.49:0/2802417587 conn(0x7f61700fbf70 msgr2=0x7f61700fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 -- 172.21.15.49:0/2802417587 shutdown_connections 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 -- 172.21.15.49:0/2802417587 wait complete. 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.938+0000 7f617590f700 1 Processor -- start 2024-03-16T08:16:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 -- start start 2024-03-16T08:16:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61701003d0 0x7f617019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6170100d90 0x7f617019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6170110120 con 0x7f61701028d0 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f617010ffa0 con 0x7f6170100d90 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f617590f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61701102a0 con 0x7f61701003d0 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36266/0 (socket says 172.21.15.49:36266) 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 -- 172.21.15.49:0/893530863 learned_addr learned my addr 172.21.15.49:0/893530863 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616effd700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61701003d0 0x7f617019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616e7fc700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6170100d90 0x7f617019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 -- 172.21.15.49:0/893530863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61701003d0 msgr2=0x7f617019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61701003d0 0x7f617019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 -- 172.21.15.49:0/893530863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6170100d90 msgr2=0x7f617019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6170100d90 0x7f617019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.939+0000 7f616f7fe700 1 -- 172.21.15.49:0/893530863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61701a2320 con 0x7f61701028d0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f616f7fe700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f616800ef60 tx=0x7f616800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61680092e0 con 0x7f61701028d0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61701a25b0 con 0x7f61701028d0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6168004bb0 con 0x7f61701028d0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168004280 con 0x7f61701028d0 2024-03-16T08:16:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.940+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a2af0 con 0x7f61701028d0 2024-03-16T08:16:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.941+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6170068f90 con 0x7f61701028d0 2024-03-16T08:16:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.942+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f616801b030 con 0x7f61701028d0 2024-03-16T08:16:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.943+0000 7f617490d700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 0x7f61580a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.943+0000 7f616effd700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 0x7f61580a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.943+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f61680ce690 con 0x7f61701028d0 2024-03-16T08:16:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.943+0000 7f616effd700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 0x7f61580a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6160007be0 tx=0x7f61600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:30.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:30.947+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6168022020 con 0x7f61701028d0 2024-03-16T08:16:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.172+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f6170062710 con 0x7f61580a2920 2024-03-16T08:16:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.173+0000 7f617490d700 1 -- 172.21.15.49:0/893530863 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f6170062710 con 0x7f61580a2920 2024-03-16T08:16:31.173 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2024-03-16T08:16:31.173 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2024-03-16T08:16:31.173 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2024-03-16T08:16:31.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 msgr2=0x7f61580a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 0x7f61580a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6160007be0 tx=0x7f61600073d0 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 msgr2=0x7f61701a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f616800ef60 tx=0x7f616800ef90 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 shutdown_connections 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f61580a2920 0x7f61580a4de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61701028d0 0x7f61701a1c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61701003d0 0x7f617019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 --2- 172.21.15.49:0/893530863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6170100d90 0x7f617019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 >> 172.21.15.49:0/893530863 conn(0x7f61700fbf70 msgr2=0x7f61700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 shutdown_connections 2024-03-16T08:16:31.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:31.175+0000 7f617590f700 1 -- 172.21.15.49:0/893530863 wait complete. 2024-03-16T08:16:31.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:31 smithi129 ceph-mon[36561]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:31.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:31 smithi154 ceph-mon[36471]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:31.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:31 smithi049 ceph-mon[31942]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:31.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-16T08:16:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:32.518 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:32 smithi049 ceph-mon[31942]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:32.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:32 smithi129 ceph-mon[36561]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:32.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:32 smithi154 ceph-mon[36471]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:33.466 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:33 smithi049 ceph-mon[31942]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:33.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:33 smithi129 ceph-mon[36561]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:33.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:33 smithi154 ceph-mon[36471]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 -- 172.21.15.49:0/754665277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48102320 msgr2=0x7efd48102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 --2- 172.21.15.49:0/754665277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48102320 0x7efd48102740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd380099c0 tx=0x7efd38009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 -- 172.21.15.49:0/754665277 shutdown_connections 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 --2- 172.21.15.49:0/754665277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4810a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 --2- 172.21.15.49:0/754665277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48102320 0x7efd48102740 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 --2- 172.21.15.49:0/754665277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48103520 0x7efd48103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 -- 172.21.15.49:0/754665277 >> 172.21.15.49:0/754665277 conn(0x7efd480fd8e0 msgr2=0x7efd480ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 -- 172.21.15.49:0/754665277 shutdown_connections 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.733+0000 7efd4f67b700 1 -- 172.21.15.49:0/754665277 wait complete. 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 Processor -- start 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 -- start start 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 0x7efd48198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810def0 con 0x7efd48103520 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810dd70 con 0x7efd48102320 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4f67b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810e070 con 0x7efd48103ec0 2024-03-16T08:16:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4cc16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4cc16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43804/0 (socket says 172.21.15.49:43804) 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.734+0000 7efd4cc16700 1 -- 172.21.15.49:0/2718238331 learned_addr learned my addr 172.21.15.49:0/2718238331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4dc18700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4d417700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 0x7efd48198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 -- 172.21.15.49:0/2718238331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 msgr2=0x7efd4819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 -- 172.21.15.49:0/2718238331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 msgr2=0x7efd48198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 0x7efd48198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 -- 172.21.15.49:0/2718238331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481a0170 con 0x7efd48103520 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4dc18700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4d417700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 0x7efd48198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4cc16700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efd4400e9d0 tx=0x7efd4400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4400cbd0 con 0x7efd48103520 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4400ee70 con 0x7efd48103520 2024-03-16T08:16:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd481a0460 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44018660 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.735+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd481a09f0 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.736+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd48068f90 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.737+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7efd44010430 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.738+0000 7efd3e7fc700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 0x7efd340a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.738+0000 7efd4d417700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 0x7efd340a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.738+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7efd440cef70 con 0x7efd48103520 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.739+0000 7efd4d417700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 0x7efd340a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd38003790 tx=0x7efd38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.741+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd4409bee0 con 0x7efd48103520 2024-03-16T08:16:33.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.956+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efd48062710 con 0x7efd340a29a0 2024-03-16T08:16:33.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.970+0000 7efd3e7fc700 1 -- 172.21.15.49:0/2718238331 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7efd48062710 con 0x7efd340a29a0 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.a smithi049 running (100s) 15s ago 99s 15.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 80728de25230 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.b smithi129 running (97s) 30s ago 96s 26.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 253f5c42bc63 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.c smithi154 running (111s) 30s ago 110s 20.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0c3ad7ae175b 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.d smithi049 running (119s) 15s ago 119s 13.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 76bfb26440d7 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.e smithi129 running (117s) 30s ago 116s 16.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b de493ff26800 2024-03-16T08:16:33.971 INFO:teuthology.orchestra.run.smithi049.stdout:mds.f smithi154 running (114s) 30s ago 113s 19.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b d354d091e09e 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.g smithi049 running (102s) 15s ago 102s 14.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 053c28ba6dba 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.h smithi129 running (108s) 30s ago 108s 20.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 51399ec5619a 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.i smithi154 running (105s) 30s ago 105s 18.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b fc38ddb5e941 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.j smithi049 running (2m) 15s ago 2m 15.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a7883aa1f873 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.k smithi129 running (2m) 30s ago 2m 13.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 880418fc63db 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mds.l smithi154 running (2m) 30s ago 2m 14.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 222958ac3969 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.x smithi049 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 3273f86bb238 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.y smithi129 *:8443,8765 running (7m) 30s ago 7m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 6c7b1da62a51 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.z smithi154 *:8443,8765 running (7m) 30s ago 7m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0305239ca55d 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a smithi049 running (9m) 15s ago 9m 49.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 54b77ba4c5c9 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b smithi129 running (7m) 30s ago 7m 37.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1775c1589efd 2024-03-16T08:16:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c smithi154 running (7m) 30s ago 7m 39.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 28848fa0265b 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 15s ago 7m 45.6M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b bbd2dcf586df 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi129 running (6m) 30s ago 6m 57.2M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b ff5b34a763e7 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi154 running (6m) 30s ago 6m 59.8M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 0b60d8089ffe 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi049 running (5m) 15s ago 5m 43.8M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5f09a2c080c8 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi129 running (5m) 30s ago 5m 52.1M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b b7b86ab39a2c 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi154 running (5m) 30s ago 5m 43.0M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 38f31445e02d 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 smithi049 running (4m) 15s ago 4m 42.7M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 64ef359ebc5f 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 smithi129 running (4m) 30s ago 4m 42.7M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b ab3a1cb9909a 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 smithi154 running (3m) 30s ago 3m 51.5M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b cbc213c89528 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 smithi049 running (3m) 15s ago 3m 44.2M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 259d62a13e60 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 smithi129 running (2m) 30s ago 2m 43.7M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 58d3a2679a72 2024-03-16T08:16:33.973 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 smithi154 running (2m) 30s ago 2m 50.5M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9c5849242124 2024-03-16T08:16:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 msgr2=0x7efd340a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 0x7efd340a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd38003790 tx=0x7efd38005fb0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 msgr2=0x7efd48198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efd4400e9d0 tx=0x7efd4400ece0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 shutdown_connections 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd340a29a0 0x7efd340a4e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd48103ec0 0x7efd4819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd48102320 0x7efd48198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 --2- 172.21.15.49:0/2718238331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48103520 0x7efd48198a50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.972+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 >> 172.21.15.49:0/2718238331 conn(0x7efd480fd8e0 msgr2=0x7efd48106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.973+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 shutdown_connections 2024-03-16T08:16:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:33.973+0000 7efd4f67b700 1 -- 172.21.15.49:0/2718238331 wait complete. 2024-03-16T08:16:34.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-16T08:16:34.893 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:35 smithi154 ceph-mon[36471]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:35.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:35 smithi154 ceph-mon[36471]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:35.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:35 smithi049 ceph-mon[31942]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:35.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:35 smithi049 ceph-mon[31942]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:35.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:35 smithi129 ceph-mon[36561]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:35.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:35 smithi129 ceph-mon[36561]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 -- 172.21.15.49:0/4055915503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4103520 msgr2=0x7f4af4103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 --2- 172.21.15.49:0/4055915503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4103520 0x7f4af4103980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009920 tx=0x7f4ae8009c30 comp rx=0 tx=0).stop 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 -- 172.21.15.49:0/4055915503 shutdown_connections 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 --2- 172.21.15.49:0/4055915503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103ec0 0x7f4af410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 --2- 172.21.15.49:0/4055915503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4102320 0x7f4af4102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 --2- 172.21.15.49:0/4055915503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4103520 0x7f4af4103980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 -- 172.21.15.49:0/4055915503 >> 172.21.15.49:0/4055915503 conn(0x7f4af40fd8e0 msgr2=0x7f4af40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 -- 172.21.15.49:0/4055915503 shutdown_connections 2024-03-16T08:16:36.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.581+0000 7f4afb621700 1 -- 172.21.15.49:0/4055915503 wait complete. 2024-03-16T08:16:36.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:36 smithi049 ceph-mon[31942]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-16T08:16:36.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.582+0000 7f4afb621700 1 Processor -- start 2024-03-16T08:16:36.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.582+0000 7f4afb621700 1 -- start start 2024-03-16T08:16:36.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.582+0000 7f4afb621700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:36.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.582+0000 7f4afb621700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 0x7f4af4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.582+0000 7f4afb621700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4103ec0 0x7f4af419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4afb621700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410df30 con 0x7f4af4103520 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4afb621700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410ddb0 con 0x7f4af4102320 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4afb621700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410e0b0 con 0x7f4af4103ec0 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af8bbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 0x7f4af4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af8bbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 0x7f4af4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43814/0 (socket says 172.21.15.49:43814) 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af8bbc700 1 -- 172.21.15.49:0/324072477 learned_addr learned my addr 172.21.15.49:0/324072477 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:57282/0 (socket says 172.21.15.49:57282) 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af9bbe700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4103ec0 0x7f4af419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 -- 172.21.15.49:0/324072477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4103ec0 msgr2=0x7f4af419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4103ec0 0x7f4af419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 -- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 msgr2=0x7f4af4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 0x7f4af4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 -- 172.21.15.49:0/324072477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af41a0260 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af93bd700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400eee0 tx=0x7f4ae400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40092e0 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae400c9c0 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af41a04f0 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.583+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4004580 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.584+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a0a80 con 0x7f4af4102320 2024-03-16T08:16:36.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.584+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af4068f90 con 0x7f4af4102320 2024-03-16T08:16:36.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.586+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4ae401b030 con 0x7f4af4102320 2024-03-16T08:16:36.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.586+0000 7f4af27fc700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 0x7f4adc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:36.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.586+0000 7f4af8bbc700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 0x7f4adc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:36.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.586+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f4ae40ce960 con 0x7f4af4102320 2024-03-16T08:16:36.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.587+0000 7f4af8bbc700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 0x7f4adc0a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009680 tx=0x7f4ae8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:36.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.593+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4ae4022020 con 0x7f4af4102320 2024-03-16T08:16:36.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:36 smithi154 ceph-mon[36471]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-16T08:16:36.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.810+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4af4062710 con 0x7f4adc0a2910 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.816+0000 7f4af27fc700 1 -- 172.21.15.49:0/324072477 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4af4062710 con 0x7f4adc0a2910 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stdout:mds.all 12/12 33s ago 2m smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 3/3 33s ago 7m smithi049=x;smithi129=y;smithi154=z;count:3 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stdout:mon 3/3 33s ago 7m smithi049:172.21.15.49=a;smithi129:172.21.15.129=b;smithi154:172.21.15.154=c;count:3 2024-03-16T08:16:36.817 INFO:teuthology.orchestra.run.smithi049.stdout:osd 12 33s ago - 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 msgr2=0x7f4adc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 0x7f4adc0a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009680 tx=0x7f4ae8019040 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 msgr2=0x7f4af4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400eee0 tx=0x7f4ae400c3a0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 shutdown_connections 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f4adc0a2910 0x7f4adc0a4dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4af4103ec0 0x7f4af419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4af4102320 0x7f4af4198600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 --2- 172.21.15.49:0/324072477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4af4103520 0x7f4af4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 >> 172.21.15.49:0/324072477 conn(0x7f4af40fd8e0 msgr2=0x7f4af4106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 shutdown_connections 2024-03-16T08:16:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:36.818+0000 7f4afb621700 1 -- 172.21.15.49:0/324072477 wait complete. 2024-03-16T08:16:36.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:36 smithi129 ceph-mon[36561]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-16T08:16:37.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-16T08:16:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:37.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:37 smithi154 ceph-mon[36471]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:37.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:37 smithi049 ceph-mon[31942]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:37.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:37 smithi129 ceph-mon[36561]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:38.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:38 smithi154 ceph-mon[36471]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:38 smithi049 ceph-mon[31942]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:38 smithi129 ceph-mon[36561]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 -- 172.21.15.49:0/1113405144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101ec0 msgr2=0x7efd68102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 --2- 172.21.15.49:0/1113405144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101ec0 0x7efd68102340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd58009990 tx=0x7efd58009ca0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 -- 172.21.15.49:0/1113405144 shutdown_connections 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 --2- 172.21.15.49:0/1113405144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101ec0 0x7efd68102340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 --2- 172.21.15.49:0/1113405144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68100ba0 0x7efd68100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 --2- 172.21.15.49:0/1113405144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68101500 0x7efd68101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 -- 172.21.15.49:0/1113405144 >> 172.21.15.49:0/1113405144 conn(0x7efd680fc760 msgr2=0x7efd680feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:39.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 -- 172.21.15.49:0/1113405144 shutdown_connections 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.367+0000 7efd6db49700 1 -- 172.21.15.49:0/1113405144 wait complete. 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 Processor -- start 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 -- start start 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68101ec0 0x7efd681a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efd68110080 con 0x7efd68101500 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810ff00 con 0x7efd68100ba0 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd6db49700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efd68110200 con 0x7efd68101ec0 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd677fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:39.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43842/0 (socket says 172.21.15.49:43842) 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd677fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:57292/0 (socket says 172.21.15.49:57292) 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd66ffd700 1 -- 172.21.15.49:0/3205709980 learned_addr learned my addr 172.21.15.49:0/3205709980 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.368+0000 7efd67fff700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68101ec0 0x7efd681a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 -- 172.21.15.49:0/3205709980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68101ec0 msgr2=0x7efd681a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68101ec0 0x7efd681a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 -- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 msgr2=0x7efd6819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 -- 172.21.15.49:0/3205709980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd681a22a0 con 0x7efd68100ba0 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd66ffd700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd677fe700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd60007ec0 tx=0x7efd6000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60004db0 con 0x7efd68100ba0 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60003a00 con 0x7efd68100ba0 2024-03-16T08:16:39.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a2590 con 0x7efd68100ba0 2024-03-16T08:16:39.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60004680 con 0x7efd68100ba0 2024-03-16T08:16:39.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.369+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a2ad0 con 0x7efd68100ba0 2024-03-16T08:16:39.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.370+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd68068f90 con 0x7efd68100ba0 2024-03-16T08:16:39.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.371+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efd6001c060 con 0x7efd68100ba0 2024-03-16T08:16:39.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.372+0000 7efd64ff9700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 0x7efd540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:39.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.372+0000 7efd66ffd700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 0x7efd540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:39.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.372+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7efd6001a070 con 0x7efd68100ba0 2024-03-16T08:16:39.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.372+0000 7efd66ffd700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 0x7efd540a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efd50011550 tx=0x7efd50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:39.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.376+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd60010070 con 0x7efd68100ba0 2024-03-16T08:16:39.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.587+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efd68062710 con 0x7efd540a2890 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.588+0000 7efd64ff9700 1 -- 172.21.15.49:0/3205709980 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efd68062710 con 0x7efd540a2890 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:smithi129 172.21.15.129 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:smithi154 172.21.15.154 2024-03-16T08:16:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:3 hosts in cluster 2024-03-16T08:16:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 msgr2=0x7efd540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 0x7efd540a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efd50011550 tx=0x7efd50009260 comp rx=0 tx=0).stop 2024-03-16T08:16:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 msgr2=0x7efd6819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd60007ec0 tx=0x7efd6000d3b0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 shutdown_connections 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efd540a2890 0x7efd540a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd68101ec0 0x7efd681a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd68100ba0 0x7efd6819a690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 --2- 172.21.15.49:0/3205709980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68101500 0x7efd6819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 >> 172.21.15.49:0/3205709980 conn(0x7efd680fc760 msgr2=0x7efd680fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 shutdown_connections 2024-03-16T08:16:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:39.590+0000 7efd6db49700 1 -- 172.21.15.49:0/3205709980 wait complete. 2024-03-16T08:16:39.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:39 smithi154 ceph-mon[36471]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:39.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:39 smithi049 ceph-mon[31942]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:39.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:39 smithi129 ceph-mon[36561]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:40.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-16T08:16:40.587 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:41.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:41 smithi154 ceph-mon[36471]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:41.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:41 smithi154 ceph-mon[36471]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:16:41.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:41 smithi049 ceph-mon[31942]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:41.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:41 smithi049 ceph-mon[31942]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:16:41.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:41 smithi129 ceph-mon[36561]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:41.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:41 smithi129 ceph-mon[36561]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T08:16:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- 172.21.15.49:0/3183696093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 msgr2=0x7febac1016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- 172.21.15.49:0/3183696093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac1016c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feba00099c0 tx=0x7feba0009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- 172.21.15.49:0/3183696093 shutdown_connections 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- 172.21.15.49:0/3183696093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- 172.21.15.49:0/3183696093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac1016c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- 172.21.15.49:0/3183696093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 0x7febac102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- 172.21.15.49:0/3183696093 >> 172.21.15.49:0/3183696093 conn(0x7febac0fc860 msgr2=0x7febac0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- 172.21.15.49:0/3183696093 shutdown_connections 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- 172.21.15.49:0/3183696093 wait complete. 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 Processor -- start 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 -- start start 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.405+0000 7febb2e37700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 0x7febac103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb2e37700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac103dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb2e37700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7febac10cdf0 con 0x7febac1012a0 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb2e37700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febac10cc70 con 0x7febac1024a0 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb2e37700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7febac10cf70 con 0x7febac102e40 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb13d4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac103dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43868/0 (socket says 172.21.15.49:43868) 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 -- 172.21.15.49:0/1837209853 learned_addr learned my addr 172.21.15.49:0/1837209853 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febabfff700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 0x7febac103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 -- 172.21.15.49:0/1837209853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 msgr2=0x7febac103dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac103dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 -- 172.21.15.49:0/1837209853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 msgr2=0x7febac103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 0x7febac103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 -- 172.21.15.49:0/1837209853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac104670 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb13d4700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb0bd3700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feba0003790 tx=0x7feba0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0009e60 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac1a2680 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0003960 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.406+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba00055e0 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.407+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1a2b10 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.407+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac068f90 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.409+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7feba0003b10 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.409+0000 7feba9ffb700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 0x7feb9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.410+0000 7febabfff700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 0x7feb9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.410+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7feba00cf8f0 con 0x7febac1012a0 2024-03-16T08:16:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.410+0000 7febabfff700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 0x7feb9c0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb98009c60 tx=0x7feb98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.413+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feba0013070 con 0x7febac1012a0 2024-03-16T08:16:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.628+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7febac0fe9f0 con 0x7feb9c0a2890 2024-03-16T08:16:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.630+0000 7feba9ffb700 1 -- 172.21.15.49:0/1837209853 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7febac0fe9f0 con 0x7feb9c0a2890 2024-03-16T08:16:42.631 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-16T08:16:42.631 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-16T08:16:42.631 INFO:teuthology.orchestra.run.smithi049.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-16T08:16:42.631 INFO:teuthology.orchestra.run.smithi049.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-16T08:16:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 msgr2=0x7feb9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 0x7feb9c0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb98009c60 tx=0x7feb98009260 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 msgr2=0x7febac105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feba0003790 tx=0x7feba0009580 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 shutdown_connections 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7feb9c0a2890 0x7feb9c0a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febac102e40 0x7febac103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febac1012a0 0x7febac105230 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 --2- 172.21.15.49:0/1837209853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febac1024a0 0x7febac103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 >> 172.21.15.49:0/1837209853 conn(0x7febac0fc860 msgr2=0x7febac0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 shutdown_connections 2024-03-16T08:16:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:42.632+0000 7febb2e37700 1 -- 172.21.15.49:0/1837209853 wait complete. 2024-03-16T08:16:43.383 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:16:43.393 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:16:43.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T08:16:43.650 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:43.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:43 smithi154 ceph-mon[36471]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:43.770 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:43 smithi049 ceph-mon[31942]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:43.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:43 smithi129 ceph-mon[36561]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:44.685 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:44 smithi049 ceph-mon[31942]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:44.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:44 smithi154 ceph-mon[36471]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:44.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:44 smithi129 ceph-mon[36561]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:16:45.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 -- 172.21.15.49:0/261960316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 msgr2=0x7fae9410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:45.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 --2- 172.21.15.49:0/261960316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9410a790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae840099c0 tx=0x7fae84009cd0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 -- 172.21.15.49:0/261960316 shutdown_connections 2024-03-16T08:16:45.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 --2- 172.21.15.49:0/261960316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9410a790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 --2- 172.21.15.49:0/261960316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94102340 0x7fae94102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 --2- 172.21.15.49:0/261960316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94103540 0x7fae941039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 -- 172.21.15.49:0/261960316 >> 172.21.15.49:0/261960316 conn(0x7fae940fd920 msgr2=0x7fae940ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 -- 172.21.15.49:0/261960316 shutdown_connections 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.309+0000 7fae9c25d700 1 -- 172.21.15.49:0/261960316 wait complete. 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 Processor -- start 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 -- start start 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 0x7fae94198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94103540 0x7fae94198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fae9410df30 con 0x7fae94103ee0 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae9410ddb0 con 0x7fae94102340 2024-03-16T08:16:45.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9c25d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae9410e0b0 con 0x7fae94103540 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58134/0 (socket says 172.21.15.49:58134) 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae997f8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94103540 0x7fae94198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae99ff9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 0x7fae94198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 -- 172.21.15.49:0/2020706491 learned_addr learned my addr 172.21.15.49:0/2020706491 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 -- 172.21.15.49:0/2020706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94103540 msgr2=0x7fae94198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94103540 0x7fae94198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 -- 172.21.15.49:0/2020706491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 msgr2=0x7fae94198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.310+0000 7fae9a7fa700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 0x7fae94198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae9a7fa700 1 -- 172.21.15.49:0/2020706491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae941a01d0 con 0x7fae94103ee0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae99ff9700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 0x7fae94198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae9a7fa700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae84009990 tx=0x7fae84009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84009e60 con 0x7fae94103ee0 2024-03-16T08:16:45.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae84003960 con 0x7fae94103ee0 2024-03-16T08:16:45.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae94074500 con 0x7fae94103ee0 2024-03-16T08:16:45.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840173f0 con 0x7fae94103ee0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.311+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae940749e0 con 0x7fae94103ee0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.312+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94068f90 con 0x7fae94103ee0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.313+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fae84003b10 con 0x7fae94103ee0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.314+0000 7fae92ffd700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 0x7fae7c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.314+0000 7fae99ff9700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 0x7fae7c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.314+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fae840cf8e0 con 0x7fae94103ee0 2024-03-16T08:16:45.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.314+0000 7fae99ff9700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 0x7fae7c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae941033a0 tx=0x7fae8c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:45.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.317+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fae84013070 con 0x7fae94103ee0 2024-03-16T08:16:45.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.567+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fae9402db70 con 0x7fae94103ee0 2024-03-16T08:16:45.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:45 smithi049 ceph-mon[31942]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:45.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.568+0000 7fae92ffd700 1 -- 172.21.15.49:0/2020706491 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3052 (secure 0 0 0) 0x7fae8409ca40 con 0x7fae94103ee0 2024-03-16T08:16:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:e23 2024-03-16T08:16:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:16:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 23 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-16T08:15:01.568993+0000 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-16T08:15:07.570394+0000 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 300 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-03-16T08:16:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 3 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1,2 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=24536,1=24505,2=24491} 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2024-03-16T08:16:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 24536 members: 24536,24491,24505 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c{0:24475} state up:standby-replay seq 1 addr [v2:172.21.15.154:6836/4234449434,v1:172.21.15.154:6837/4234449434] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.i{1:24505} state up:active seq 6 addr [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.154:6834/2047612746,v1:172.21.15.154:6835/2047612746] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.h{2:24491} state up:active seq 7 addr [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e{2:24461} state up:standby-replay seq 1 addr [v2:172.21.15.129:6834/272945510,v1:172.21.15.129:6835/272945510] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.49:6834/2399676198,v1:172.21.15.49:6835/2399676198] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.49:6836/761407043,v1:172.21.15.49:6837/761407043] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.573 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.g{-1:14616} state up:standby seq 1 addr [v2:172.21.15.49:6838/2351115602,v1:172.21.15.49:6839/2351115602] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.573 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a{-1:14628} state up:standby seq 1 addr [v2:172.21.15.49:6840/1594522521,v1:172.21.15.49:6841/1594522521] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.573 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.154:6832/866170540,v1:172.21.15.154:6833/866170540] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.573 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.k{-1:24422} state up:standby seq 1 addr [v2:172.21.15.129:6832/3770218709,v1:172.21.15.129:6833/3770218709] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 msgr2=0x7fae7c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 0x7fae7c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae941033a0 tx=0x7fae8c00b040 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 msgr2=0x7fae9419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae84009990 tx=0x7fae84009580 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 shutdown_connections 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fae7c0a2890 0x7fae7c0a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae94103ee0 0x7fae9419fb30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae94102340 0x7fae94198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 --2- 172.21.15.49:0/2020706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae94103540 0x7fae94198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 >> 172.21.15.49:0/2020706491 conn(0x7fae940fd920 msgr2=0x7fae94106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:45.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 shutdown_connections 2024-03-16T08:16:45.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:45.571+0000 7fae9c25d700 1 -- 172.21.15.49:0/2020706491 wait complete. 2024-03-16T08:16:45.575 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:45.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:45 smithi154 ceph-mon[36471]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:45.760 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:45 smithi129 ceph-mon[36561]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:46.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-16T08:16:46.453 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:46 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2020706491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:16:46.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:46 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2020706491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:16:46.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:46 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2020706491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:16:47.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:47 smithi049 ceph-mon[31942]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:47.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:47 smithi154 ceph-mon[36471]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:47.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:47 smithi129 ceph-mon[36561]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 -- 172.21.15.49:0/140140074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102da0 msgr2=0x7fe68c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 --2- 172.21.15.49:0/140140074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102da0 0x7fe68c109650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe68400b0d0 tx=0x7fe68400b3e0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 -- 172.21.15.49:0/140140074 shutdown_connections 2024-03-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 --2- 172.21.15.49:0/140140074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102da0 0x7fe68c109650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 --2- 172.21.15.49:0/140140074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 0x7fe68c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 --2- 172.21.15.49:0/140140074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102400 0x7fe68c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 -- 172.21.15.49:0/140140074 >> 172.21.15.49:0/140140074 conn(0x7fe68c0fc7a0 msgr2=0x7fe68c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 -- 172.21.15.49:0/140140074 shutdown_connections 2024-03-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 -- 172.21.15.49:0/140140074 wait complete. 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.108+0000 7fe690d48700 1 Processor -- start 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 -- start start 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 0x7fe68c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 0x7fe68c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10ce00 con 0x7fe68c102400 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10cc80 con 0x7fe68c101200 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe690d48700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10cf80 con 0x7fe68c102da0 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58150/0 (socket says 172.21.15.49:58150) 2024-03-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 -- 172.21.15.49:0/2600930368 learned_addr learned my addr 172.21.15.49:0/2600930368 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe68a59c700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 0x7fe68c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe68ad9d700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 0x7fe68c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 -- 172.21.15.49:0/2600930368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 msgr2=0x7fe68c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 0x7fe68c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 -- 172.21.15.49:0/2600930368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 msgr2=0x7fe68c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 0x7fe68c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 -- 172.21.15.49:0/2600930368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c1a2370 con 0x7fe68c102400 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe68ad9d700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 0x7fe68c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.109+0000 7fe689d9b700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe68000e9f0 tx=0x7fe68000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.110+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68000cbd0 con 0x7fe68c102400 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.110+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68000ee90 con 0x7fe68c102400 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.110+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800056c0 con 0x7fe68c102400 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.110+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c1a2660 con 0x7fe68c102400 2024-03-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.110+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a2b50 con 0x7fe68c102400 2024-03-16T08:16:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.111+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c068f90 con 0x7fe68c102400 2024-03-16T08:16:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.112+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe680004750 con 0x7fe68c102400 2024-03-16T08:16:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.112+0000 7fe67b7fe700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 0x7fe6740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:48.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.113+0000 7fe68a59c700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 0x7fe6740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:48.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.113+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe6800ceee0 con 0x7fe68c102400 2024-03-16T08:16:48.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.113+0000 7fe68a59c700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 0x7fe6740a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe68c102260 tx=0x7fe67c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.116+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe680053740 con 0x7fe68c102400 2024-03-16T08:16:48.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.320+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe68c1a2e10 con 0x7fe68c102400 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.321+0000 7fe67b7fe700 1 -- 172.21.15.49:0/2600930368 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5250 (secure 0 0 0) 0x7fe68009bfb0 con 0x7fe68c102400 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 88 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-16T08:07:09.044882+0000 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-16T08:16:05.377762+0000 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-16T08:16:48.322 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 26 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client luminous 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release squid 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 12 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6802/3756414183,v1:172.21.15.49:6803/3756414183] [v2:172.21.15.49:6804/3756414183,v1:172.21.15.49:6805/3756414183] exists,up 7e333c72-baee-40c0-a078-372ada0c1511 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/646273559,v1:172.21.15.129:6801/646273559] [v2:172.21.15.129:6802/646273559,v1:172.21.15.129:6803/646273559] exists,up 79ddf11d-d24d-4221-9317-a912d69ff440 2024-03-16T08:16:48.323 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/487578356,v1:172.21.15.154:6801/487578356] [v2:172.21.15.154:6802/487578356,v1:172.21.15.154:6803/487578356] exists,up 77bcc416-5d33-4e76-bd0b-7b12aceb1a27 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6810/1033582524,v1:172.21.15.49:6811/1033582524] [v2:172.21.15.49:6812/1033582524,v1:172.21.15.49:6813/1033582524] exists,up f6b81081-263e-4e72-8499-aea94ed43b50 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/615229289,v1:172.21.15.129:6809/615229289] [v2:172.21.15.129:6810/615229289,v1:172.21.15.129:6811/615229289] exists,up a99b2da5-a3b6-423d-9884-497aae8108de 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/4188101354,v1:172.21.15.154:6809/4188101354] [v2:172.21.15.154:6810/4188101354,v1:172.21.15.154:6811/4188101354] exists,up 9f5563fc-b860-41fd-98a7-5cabd7bb92c9 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6818/3010410053,v1:172.21.15.49:6819/3010410053] [v2:172.21.15.49:6820/3010410053,v1:172.21.15.49:6821/3010410053] exists,up 36672b95-4a0b-4ffc-9103-5f9e2c187b9b 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/252652011,v1:172.21.15.129:6817/252652011] [v2:172.21.15.129:6818/252652011,v1:172.21.15.129:6819/252652011] exists,up 313db83e-0f1f-4748-b054-77708bd6a27e 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/160187571,v1:172.21.15.154:6817/160187571] [v2:172.21.15.154:6818/160187571,v1:172.21.15.154:6819/160187571] exists,up c82b22c1-18d5-40dd-8a86-428405308e1a 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6826/1471236265,v1:172.21.15.49:6827/1471236265] [v2:172.21.15.49:6828/1471236265,v1:172.21.15.49:6829/1471236265] exists,up 693d0e23-6cdd-41f2-b7c4-f9bb3686edbf 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/2725967668,v1:172.21.15.129:6825/2725967668] [v2:172.21.15.129:6826/2725967668,v1:172.21.15.129:6827/2725967668] exists,up 7fdf3874-86b2-4175-a20c-41ac7241c806 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/901406241,v1:172.21.15.154:6825/901406241] [v2:172.21.15.154:6826/901406241,v1:172.21.15.154:6827/901406241] exists,up c9663681-0e97-4339-931e-27dc3f68bb58 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.4 [2,11] 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.f [8,11] 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.3 [4,9] 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.1e [3,9,5,6] 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.27 [4,8] 2024-03-16T08:16:48.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.31 [4,7] 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.33 [5,9] 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/3066712091 expires 2024-03-17T08:08:03.498489+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/372564765 expires 2024-03-17T08:08:03.498489+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1466047614 expires 2024-03-17T08:07:36.305451+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1726594106 expires 2024-03-17T08:07:36.305451+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/2659026035 expires 2024-03-17T08:08:03.498489+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/1374126004 expires 2024-03-17T08:07:36.305451+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/1374126004 expires 2024-03-17T08:07:36.305451+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/3199936220 expires 2024-03-17T08:07:36.305451+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/2659026035 expires 2024-03-17T08:08:03.498489+0000 2024-03-16T08:16:48.325 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/3531600398 expires 2024-03-17T08:08:03.498489+0000 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 msgr2=0x7fe6740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 0x7fe6740a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe68c102260 tx=0x7fe67c009260 comp rx=0 tx=0).stop 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 msgr2=0x7fe68c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe68000e9f0 tx=0x7fe68000ed00 comp rx=0 tx=0).stop 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 shutdown_connections 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe6740a2890 0x7fe6740a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe68c102da0 0x7fe68c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe68c101200 0x7fe68c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 --2- 172.21.15.49:0/2600930368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c102400 0x7fe68c19ac50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:48.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 >> 172.21.15.49:0/2600930368 conn(0x7fe68c0fc7a0 msgr2=0x7fe68c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:48.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 shutdown_connections 2024-03-16T08:16:48.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:48.324+0000 7fe690d48700 1 -- 172.21.15.49:0/2600930368 wait complete. 2024-03-16T08:16:48.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:48 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2600930368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T08:16:48.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:48 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2600930368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T08:16:48.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:48 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2600930368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T08:16:49.043 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-16T08:16:49.054 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:49.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 -- 172.21.15.49:0/2739164854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 msgr2=0x7f9c04108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/2739164854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c04108b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009ca0 tx=0x7f9bfc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 -- 172.21.15.49:0/2739164854 shutdown_connections 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/2739164854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c04108b40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/2739164854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c041018f0 0x7f9c04101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/2739164854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c04100700 0x7f9c04100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 -- 172.21.15.49:0/2739164854 >> 172.21.15.49:0/2739164854 conn(0x7f9c040fbc70 msgr2=0x7f9c040fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 -- 172.21.15.49:0/2739164854 shutdown_connections 2024-03-16T08:16:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.164+0000 7f9c0bb60640 1 -- 172.21.15.49:0/2739164854 wait complete. 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 Processor -- start 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 -- start start 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c04100700 0x7f9c04197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c041018f0 0x7f9c04198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410c380 con 0x7f9c041022b0 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410c200 con 0x7f9c041018f0 2024-03-16T08:16:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0bb60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410c500 con 0x7f9c04100700 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58156/0 (socket says 172.21.15.49:58156) 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c098fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c04100700 0x7f9c04197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 -- 172.21.15.49:0/1197892241 learned_addr learned my addr 172.21.15.49:0/1197892241 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c090fb640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c041018f0 0x7f9c04198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 -- 172.21.15.49:0/1197892241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c04100700 msgr2=0x7f9c04197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c04100700 0x7f9c04197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 -- 172.21.15.49:0/1197892241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c041018f0 msgr2=0x7f9c04198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c041018f0 0x7f9c04198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.165+0000 7f9c0a0fd640 1 -- 172.21.15.49:0/1197892241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0419f6f0 con 0x7f9c041022b0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9c0a0fd640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc037040 tx=0x7f9bfc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc041020 con 0x7f9c041022b0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0419f980 con 0x7f9c041022b0 2024-03-16T08:16:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bfc034cd0 con 0x7f9c041022b0 2024-03-16T08:16:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc039040 con 0x7f9c041022b0 2024-03-16T08:16:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.166+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0419fec0 con 0x7f9c041022b0 2024-03-16T08:16:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.167+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c041a00f0 con 0x7f9c041022b0 2024-03-16T08:16:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.168+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9bfc044020 con 0x7f9c041022b0 2024-03-16T08:16:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.169+0000 7f9bf6ffd640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 0x7f9bf00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.169+0000 7f9c098fc640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 0x7f9bf00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.169+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f9bfc0f0850 con 0x7f9c041022b0 2024-03-16T08:16:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.169+0000 7f9c098fc640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 0x7f9bf00a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c04101750 tx=0x7f9bf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.173+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9bfc04b020 con 0x7f9c041022b0 2024-03-16T08:16:49.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.358+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c041a0480 con 0x7f9c041022b0 2024-03-16T08:16:49.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.360+0000 7f9bf6ffd640 1 -- 172.21.15.49:0/1197892241 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7f9c041a0480 con 0x7f9c041022b0 2024-03-16T08:16:49.361 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:49.362 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:49.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 msgr2=0x7f9bf00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 0x7f9bf00a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c04101750 tx=0x7f9bf8005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 msgr2=0x7f9c0419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc037040 tx=0x7f9bfc032f40 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 shutdown_connections 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f9bf00a2800 0x7f9bf00a4cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c041022b0 0x7f9c0419f080 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c041018f0 0x7f9c04198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 --2- 172.21.15.49:0/1197892241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c04100700 0x7f9c04197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 >> 172.21.15.49:0/1197892241 conn(0x7f9c040fbc70 msgr2=0x7f9c04068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 shutdown_connections 2024-03-16T08:16:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.362+0000 7f9c0bb60640 1 -- 172.21.15.49:0/1197892241 wait complete. 2024-03-16T08:16:49.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:16:49.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:16:49.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 -- 172.21.15.49:0/3549716904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd738100700 msgr2=0x7fd738100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 --2- 172.21.15.49:0/3549716904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd738100700 0x7fd738100b20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd728009900 tx=0x7fd72802bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 -- 172.21.15.49:0/3549716904 shutdown_connections 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 --2- 172.21.15.49:0/3549716904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 0x7fd738108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 --2- 172.21.15.49:0/3549716904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7381018f0 0x7fd738101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 --2- 172.21.15.49:0/3549716904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd738100700 0x7fd738100b20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 -- 172.21.15.49:0/3549716904 >> 172.21.15.49:0/3549716904 conn(0x7fd7380fbc90 msgr2=0x7fd7380fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 -- 172.21.15.49:0/3549716904 shutdown_connections 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.485+0000 7fd73d012640 1 -- 172.21.15.49:0/3549716904 wait complete. 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 Processor -- start 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 -- start start 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd738100700 0x7fd738104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 0x7fd738103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810c370 con 0x7fd7381018f0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810c1f0 con 0x7fd7381022b0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73d012640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810c4f0 con 0x7fd738100700 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd72ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd72ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58174/0 (socket says 172.21.15.49:58174) 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd72ffff640 1 -- 172.21.15.49:0/1987827996 learned_addr learned my addr 172.21.15.49:0/1987827996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd736d9d640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd738100700 0x7fd738104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd73759e640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 0x7fd738103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd72ffff640 1 -- 172.21.15.49:0/1987827996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd738100700 msgr2=0x7fd738104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.486+0000 7fd72ffff640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd738100700 0x7fd738104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd72ffff640 1 -- 172.21.15.49:0/1987827996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 msgr2=0x7fd738103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd72ffff640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 0x7fd738103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd72ffff640 1 -- 172.21.15.49:0/1987827996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738103aa0 con 0x7fd7381018f0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd72ffff640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e9f0 tx=0x7fd72000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72000cd80 con 0x7fd7381018f0 2024-03-16T08:16:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7381a1cb0 con 0x7fd7381018f0 2024-03-16T08:16:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd720005500 con 0x7fd7381018f0 2024-03-16T08:16:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720010690 con 0x7fd7381018f0 2024-03-16T08:16:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.487+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7381a21b0 con 0x7fd7381018f0 2024-03-16T08:16:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.488+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd738068a20 con 0x7fd7381018f0 2024-03-16T08:16:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.489+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd7200046c0 con 0x7fd7381018f0 2024-03-16T08:16:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.489+0000 7fd734d99640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 0x7fd7240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.490+0000 7fd736d9d640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 0x7fd7240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.490+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fd7200ceeb0 con 0x7fd7381018f0 2024-03-16T08:16:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.490+0000 7fd736d9d640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 0x7fd7240a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd728000c00 tx=0x7fd728005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.494+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd72009bf80 con 0x7fd7381018f0 2024-03-16T08:16:49.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.677+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7381a2b20 con 0x7fd7381018f0 2024-03-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.678+0000 7fd734d99640 1 -- 172.21.15.49:0/1987827996 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd72009b880 con 0x7fd7381018f0 2024-03-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 msgr2=0x7fd7240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 0x7fd7240a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd728000c00 tx=0x7fd728005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 msgr2=0x7fd738102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e9f0 tx=0x7fd72000edf0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 shutdown_connections 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7381022b0 0x7fd738103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fd7240a2800 0x7fd7240a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7381018f0 0x7fd738102cd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 --2- 172.21.15.49:0/1987827996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd738100700 0x7fd738104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 >> 172.21.15.49:0/1987827996 conn(0x7fd7380fbc90 msgr2=0x7fd7380fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 shutdown_connections 2024-03-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.680+0000 7fd73d012640 1 -- 172.21.15.49:0/1987827996 wait complete. 2024-03-16T08:16:49.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:49 smithi049 ceph-mon[31942]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:49.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:49 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1197892241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:49.692 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:16:49.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:49 smithi154 ceph-mon[36471]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:49.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:49 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1197892241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 -- 172.21.15.49:0/521622287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 msgr2=0x7efc281082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 --2- 172.21.15.49:0/521622287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc281082d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc20009ca0 tx=0x7efc2002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 -- 172.21.15.49:0/521622287 shutdown_connections 2024-03-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 --2- 172.21.15.49:0/521622287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc281082d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 --2- 172.21.15.49:0/521622287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc28100ff0 0x7efc28101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 --2- 172.21.15.49:0/521622287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc280ffea0 0x7efc281002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 -- 172.21.15.49:0/521622287 >> 172.21.15.49:0/521622287 conn(0x7efc280fb410 msgr2=0x7efc280fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 -- 172.21.15.49:0/521622287 shutdown_connections 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.802+0000 7efc2fd29640 1 -- 172.21.15.49:0/521622287 wait complete. 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 Processor -- start 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 -- start start 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc28100ff0 0x7efc28198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc2810d500 con 0x7efc281019b0 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc2810d380 con 0x7efc280ffea0 2024-03-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2fd29640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efc2810d680 con 0x7efc28100ff0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2dac5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2dac5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:44404/0 (socket says 172.21.15.49:44404) 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58200/0 (socket says 172.21.15.49:58200) 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2dac5640 1 -- 172.21.15.49:0/976500516 learned_addr learned my addr 172.21.15.49:0/976500516 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2d2c4640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc28100ff0 0x7efc28198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 -- 172.21.15.49:0/976500516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc28100ff0 msgr2=0x7efc28198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc28100ff0 0x7efc28198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 -- 172.21.15.49:0/976500516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 msgr2=0x7efc28197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2e2c6640 1 -- 172.21.15.49:0/976500516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2819f700 con 0x7efc281019b0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.803+0000 7efc2dac5640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc2e2c6640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc2002c630 tx=0x7efc20032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20041020 con 0x7efc281019b0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2819f990 con 0x7efc281019b0 2024-03-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20034cd0 con 0x7efc281019b0 2024-03-16T08:16:49.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20039040 con 0x7efc281019b0 2024-03-16T08:16:49.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.804+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2819fea0 con 0x7efc281019b0 2024-03-16T08:16:49.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.805+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc28107150 con 0x7efc281019b0 2024-03-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.806+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efc20044020 con 0x7efc281019b0 2024-03-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.806+0000 7efc1effd640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 0x7efc140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.806+0000 7efc2dac5640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 0x7efc140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.807+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7efc200f07d0 con 0x7efc281019b0 2024-03-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.807+0000 7efc2dac5640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 0x7efc140a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efc24011550 tx=0x7efc24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:49.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.811+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc2004b020 con 0x7efc281019b0 2024-03-16T08:16:49.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:49 smithi129 ceph-mon[36561]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:16:49.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:49 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1197892241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:49.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.994+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efc280623e0 con 0x7efc281019b0 2024-03-16T08:16:49.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.995+0000 7efc1effd640 1 -- 172.21.15.49:0/976500516 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7efc200bd930 con 0x7efc281019b0 2024-03-16T08:16:49.996 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 msgr2=0x7efc140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 0x7efc140a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efc24011550 tx=0x7efc24005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 msgr2=0x7efc2819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc2002c630 tx=0x7efc20032f20 comp rx=0 tx=0).stop 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 shutdown_connections 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efc140a2800 0x7efc140a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc281019b0 0x7efc2819f090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc28100ff0 0x7efc28198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 --2- 172.21.15.49:0/976500516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc280ffea0 0x7efc28197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:49.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 >> 172.21.15.49:0/976500516 conn(0x7efc280fb410 msgr2=0x7efc28104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:49.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 shutdown_connections 2024-03-16T08:16:49.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:49.997+0000 7efc2fd29640 1 -- 172.21.15.49:0/976500516 wait complete. 2024-03-16T08:16:50.009 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:16:50.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 -- 172.21.15.49:0/3385932737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc01096c0 msgr2=0x7fadc010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 --2- 172.21.15.49:0/3385932737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc01096c0 0x7fadc010bb50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009ca0 tx=0x7fadb802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 -- 172.21.15.49:0/3385932737 shutdown_connections 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 --2- 172.21.15.49:0/3385932737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc01096c0 0x7fadc010bb50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 --2- 172.21.15.49:0/3385932737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc0074240 0x7fadc00746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 --2- 172.21.15.49:0/3385932737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0075820 0x7fadc0073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 -- 172.21.15.49:0/3385932737 >> 172.21.15.49:0/3385932737 conn(0x7fadc00fcd70 msgr2=0x7fadc00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 -- 172.21.15.49:0/3385932737 shutdown_connections 2024-03-16T08:16:50.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.118+0000 7fadc71b3640 1 -- 172.21.15.49:0/3385932737 wait complete. 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 Processor -- start 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 -- start start 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 0x7fadc0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 0x7fadc01a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010f660 con 0x7fadc0075820 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010f4e0 con 0x7fadc0074240 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc71b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010f7e0 con 0x7fadc01096c0 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc4f4f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 0x7fadc0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58216/0 (socket says 172.21.15.49:58216) 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadbffff640 1 -- 172.21.15.49:0/772959172 learned_addr learned my addr 172.21.15.49:0/772959172 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.119+0000 7fadc5750640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 0x7fadc01a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 -- 172.21.15.49:0/772959172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 msgr2=0x7fadc01a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 0x7fadc01a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 -- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 msgr2=0x7fadc019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 -- 172.21.15.49:0/772959172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01a18e0 con 0x7fadc0074240 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc5750640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 0x7fadc01a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadbffff640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc4f4f640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 0x7fadc0199ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e9f0 tx=0x7fadb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400cd80 con 0x7fadc0074240 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01a1bd0 con 0x7fadc0074240 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb40035f0 con 0x7fadc0074240 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4010670 con 0x7fadc0074240 2024-03-16T08:16:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.120+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01a2170 con 0x7fadc0074240 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.121+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc01083f0 con 0x7fadc0074240 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.122+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fadb40037a0 con 0x7fadc0074240 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.123+0000 7fadbdffb640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 0x7fadb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.123+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fadb40cecd0 con 0x7fadc0074240 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.123+0000 7fadbffff640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 0x7fadb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.123+0000 7fadbffff640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 0x7fadb00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadc019b1a0 tx=0x7fadac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.127+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fadb409bda0 con 0x7fadc0074240 2024-03-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.311+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fadc00623e0 con 0x7fadc0074240 2024-03-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.312+0000 7fadbdffb640 1 -- 172.21.15.49:0/772959172 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fadb409b6a0 con 0x7fadc0074240 2024-03-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 msgr2=0x7fadb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 0x7fadb00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadc019b1a0 tx=0x7fadac005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 msgr2=0x7fadc0199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 0x7fadc0199ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e9f0 tx=0x7fadb400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 shutdown_connections 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadc01096c0 0x7fadc01a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fadb00a2800 0x7fadb00a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadc0075820 0x7fadc019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 --2- 172.21.15.49:0/772959172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadc0074240 0x7fadc0199ca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 >> 172.21.15.49:0/772959172 conn(0x7fadc00fcd70 msgr2=0x7fadc01055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 shutdown_connections 2024-03-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.314+0000 7fadc71b3640 1 -- 172.21.15.49:0/772959172 wait complete. 2024-03-16T08:16:50.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- 172.21.15.49:0/4876221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348101810 msgr2=0x7fe348101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 --2- 172.21.15.49:0/4876221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348101810 0x7fe348101c30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009ca0 tx=0x7fe33c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- 172.21.15.49:0/4876221 shutdown_connections 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 --2- 172.21.15.49:0/4876221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 0x7fe348109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 --2- 172.21.15.49:0/4876221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348102a00 0x7fe348102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 --2- 172.21.15.49:0/4876221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348101810 0x7fe348101c30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- 172.21.15.49:0/4876221 >> 172.21.15.49:0/4876221 conn(0x7fe3480fcdc0 msgr2=0x7fe3480ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- 172.21.15.49:0/4876221 shutdown_connections 2024-03-16T08:16:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- 172.21.15.49:0/4876221 wait complete. 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 Processor -- start 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.436+0000 7fe34cb37640 1 -- start start 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 0x7fe348197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 0x7fe34819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810d460 con 0x7fe348102a00 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810d2e0 con 0x7fe3481033c0 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810d5e0 con 0x7fe348101810 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58230/0 (socket says 172.21.15.49:58230) 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 0x7fe348197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 -- 172.21.15.49:0/2112578384 learned_addr learned my addr 172.21.15.49:0/2112578384 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe346d9d640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 0x7fe34819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 -- 172.21.15.49:0/2112578384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 msgr2=0x7fe348197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 0x7fe348197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 -- 172.21.15.49:0/2112578384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 msgr2=0x7fe34819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 0x7fe34819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 -- 172.21.15.49:0/2112578384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34819f720 con 0x7fe348102a00 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34659c640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 0x7fe348197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe345d9b640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33800cd80 con 0x7fe348102a00 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34819fa10 con 0x7fe348102a00 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe338010910 con 0x7fe348102a00 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.437+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380188f0 con 0x7fe348102a00 2024-03-16T08:16:50.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.438+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34819ffb0 con 0x7fe348102a00 2024-03-16T08:16:50.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.438+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe348108bb0 con 0x7fe348102a00 2024-03-16T08:16:50.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.440+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe338010ac0 con 0x7fe348102a00 2024-03-16T08:16:50.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.440+0000 7fe3377fe640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 0x7fe3300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.440+0000 7fe34659c640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 0x7fe3300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.440+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe3380cf1f0 con 0x7fe348102a00 2024-03-16T08:16:50.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.441+0000 7fe34659c640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 0x7fe3300a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe33c02c700 tx=0x7fe33c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.444+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe33809bc70 con 0x7fe348102a00 2024-03-16T08:16:50.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.669+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe34802d1b0 con 0x7fe348102a00 2024-03-16T08:16:50.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.669+0000 7fe3377fe640 1 -- 172.21.15.49:0/2112578384 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7fe33809fd10 con 0x7fe348102a00 2024-03-16T08:16:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:50.671 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:16:50.671 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 msgr2=0x7fe3300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 0x7fe3300a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe33c02c700 tx=0x7fe33c032590 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 msgr2=0x7fe348198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9f0 tx=0x7fe33800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 shutdown_connections 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3481033c0 0x7fe34819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3300a2800 0x7fe3300a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe348102a00 0x7fe348198040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 --2- 172.21.15.49:0/2112578384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe348101810 0x7fe348197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 >> 172.21.15.49:0/2112578384 conn(0x7fe3480fcdc0 msgr2=0x7fe348105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 shutdown_connections 2024-03-16T08:16:50.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.671+0000 7fe34cb37640 1 -- 172.21.15.49:0/2112578384 wait complete. 2024-03-16T08:16:50.683 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:50 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1987827996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.683 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:50 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/976500516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.683 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:50 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/772959172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.684 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:50 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1987827996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:50 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/976500516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:50 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/772959172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 -- 172.21.15.49:0/2386053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c101970 msgr2=0x7f499c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 --2- 172.21.15.49:0/2386053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c101970 0x7f499c101df0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4998009900 tx=0x7f499802bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 -- 172.21.15.49:0/2386053299 shutdown_connections 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 --2- 172.21.15.49:0/2386053299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c102330 0x7f499c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 --2- 172.21.15.49:0/2386053299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c101970 0x7f499c101df0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 --2- 172.21.15.49:0/2386053299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c100780 0x7f499c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 -- 172.21.15.49:0/2386053299 >> 172.21.15.49:0/2386053299 conn(0x7f499c0fbd10 msgr2=0x7f499c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 -- 172.21.15.49:0/2386053299 shutdown_connections 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.791+0000 7f49a3c12640 1 -- 172.21.15.49:0/2386053299 wait complete. 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 Processor -- start 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 -- start start 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c101970 0x7f499c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c102330 0x7f499c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10d4b0 con 0x7f499c100780 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10d330 con 0x7f499c101970 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a3c12640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10d630 con 0x7f499c102330 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58246/0 (socket says 172.21.15.49:58246) 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 -- 172.21.15.49:0/212064490 learned_addr learned my addr 172.21.15.49:0/212064490 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a11ad640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c101970 0x7f499c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a21af640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c102330 0x7f499c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 -- 172.21.15.49:0/212064490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c102330 msgr2=0x7f499c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c102330 0x7f499c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 -- 172.21.15.49:0/212064490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c101970 msgr2=0x7f499c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c101970 0x7f499c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 -- 172.21.15.49:0/212064490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c19f720 con 0x7f499c100780 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.792+0000 7f49a19ae640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f499000e9c0 tx=0x7f499000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499000cd80 con 0x7f499c100780 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c19fa10 con 0x7f499c100780 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49900035f0 con 0x7f499c100780 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990010690 con 0x7f499c100780 2024-03-16T08:16:50.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c19ff30 con 0x7f499c100780 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.793+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c107aa0 con 0x7f499c100780 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.795+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f49900037a0 con 0x7f499c100780 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.795+0000 7f498effd640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.795+0000 7f49a11ad640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.795+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f49900cee60 con 0x7f499c100780 2024-03-16T08:16:50.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.796+0000 7f49a11ad640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 0x7f49880a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f499802c7b0 tx=0x7f4998005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:50.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.799+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f499009bf30 con 0x7f499c100780 2024-03-16T08:16:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:50 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1987827996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:50 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/976500516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:50 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/772959172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:50.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.984+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f499c02d1b0 con 0x7f499c100780 2024-03-16T08:16:50.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.985+0000 7f498effd640 1 -- 172.21.15.49:0/212064490 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7f499009b830 con 0x7f499c100780 2024-03-16T08:16:50.987 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:50.987 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 msgr2=0x7f49880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 0x7f49880a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f499802c7b0 tx=0x7f4998005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 msgr2=0x7f499c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f499000e9c0 tx=0x7f499000ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 shutdown_connections 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f499c102330 0x7f499c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499c101970 0x7f499c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 --2- 172.21.15.49:0/212064490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499c100780 0x7f499c197b00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.987+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 >> 172.21.15.49:0/212064490 conn(0x7f499c0fbd10 msgr2=0x7f499c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:50.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.988+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 shutdown_connections 2024-03-16T08:16:50.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:50.988+0000 7f49a3c12640 1 -- 172.21.15.49:0/212064490 wait complete. 2024-03-16T08:16:51.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:16:51.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:16:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 -- 172.21.15.49:0/1312319502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84100700 msgr2=0x7fee84100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1312319502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84100700 0x7fee84100b20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee74009900 tx=0x7fee7402bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 -- 172.21.15.49:0/1312319502 shutdown_connections 2024-03-16T08:16:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1312319502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841022b0 0x7fee84108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1312319502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee841018f0 0x7fee84101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1312319502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84100700 0x7fee84100b20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 -- 172.21.15.49:0/1312319502 >> 172.21.15.49:0/1312319502 conn(0x7fee840fbc90 msgr2=0x7fee840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 -- 172.21.15.49:0/1312319502 shutdown_connections 2024-03-16T08:16:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.110+0000 7fee88fc4640 1 -- 172.21.15.49:0/1312319502 wait complete. 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 Processor -- start 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 -- start start 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 0x7fee8406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 0x7fee8419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee8410c350 con 0x7fee841022b0 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee8410c1d0 con 0x7fee84100700 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee88fc4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee8410c4d0 con 0x7fee841018f0 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee8259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 0x7fee8406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee82d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 0x7fee8419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee8259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 0x7fee8406a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:44490/0 (socket says 172.21.15.49:44490) 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee8259c640 1 -- 172.21.15.49:0/1856971178 learned_addr learned my addr 172.21.15.49:0/1856971178 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee82d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58266/0 (socket says 172.21.15.49:58266) 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 -- 172.21.15.49:0/1856971178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 msgr2=0x7fee8406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 0x7fee8406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 -- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 msgr2=0x7fee8419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee81d9b640 1 -- 172.21.15.49:0/1856971178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8419f7d0 con 0x7fee841018f0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.111+0000 7fee82d9d640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee81d9b640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 0x7fee8419c8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9f0 tx=0x7fee7800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800ccc0 con 0x7fee841018f0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8419fa60 con 0x7fee841018f0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee78010910 con 0x7fee841018f0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78018790 con 0x7fee841018f0 2024-03-16T08:16:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.112+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8419ffb0 con 0x7fee841018f0 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.113+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee84107af0 con 0x7fee841018f0 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.114+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fee78010ac0 con 0x7fee841018f0 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.114+0000 7fee737fe640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 0x7fee6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.114+0000 7fee8259c640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 0x7fee6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.115+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fee780ced00 con 0x7fee841018f0 2024-03-16T08:16:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.115+0000 7fee8259c640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 0x7fee6c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fee7402c8e0 tx=0x7fee74005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.118+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fee7809bdd0 con 0x7fee841018f0 2024-03-16T08:16:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.301+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee8402d400 con 0x7fee841018f0 2024-03-16T08:16:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.302+0000 7fee737fe640 1 -- 172.21.15.49:0/1856971178 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fee7809b6d0 con 0x7fee841018f0 2024-03-16T08:16:51.303 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 msgr2=0x7fee6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 0x7fee6c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fee7402c8e0 tx=0x7fee74005b50 comp rx=0 tx=0).stop 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 msgr2=0x7fee8419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 0x7fee8419c8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9f0 tx=0x7fee7800edf0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 shutdown_connections 2024-03-16T08:16:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fee6c0a2800 0x7fee6c0a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee841022b0 0x7fee8419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee841018f0 0x7fee8419c8a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 --2- 172.21.15.49:0/1856971178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee84100700 0x7fee8406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 >> 172.21.15.49:0/1856971178 conn(0x7fee840fbc90 msgr2=0x7fee84104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 shutdown_connections 2024-03-16T08:16:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.304+0000 7fee88fc4640 1 -- 172.21.15.49:0/1856971178 wait complete. 2024-03-16T08:16:51.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:16:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 -- 172.21.15.49:0/2718208631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7581022b0 msgr2=0x7fe758108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 --2- 172.21.15.49:0/2718208631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7581022b0 0x7fe758108b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe750009ca0 tx=0x7fe75002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 -- 172.21.15.49:0/2718208631 shutdown_connections 2024-03-16T08:16:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 --2- 172.21.15.49:0/2718208631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7581022b0 0x7fe758108b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 --2- 172.21.15.49:0/2718208631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 0x7fe758101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 --2- 172.21.15.49:0/2718208631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe758100700 0x7fe758100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 -- 172.21.15.49:0/2718208631 >> 172.21.15.49:0/2718208631 conn(0x7fe7580fbc90 msgr2=0x7fe7580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 -- 172.21.15.49:0/2718208631 shutdown_connections 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.426+0000 7fe75f885640 1 -- 172.21.15.49:0/2718208631 wait complete. 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 Processor -- start 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 -- start start 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 0x7fe75819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581022b0 0x7fe7581a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810c380 con 0x7fe758100700 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810c200 con 0x7fe7581018f0 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75f885640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810c500 con 0x7fe7581022b0 2024-03-16T08:16:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58286/0 (socket says 172.21.15.49:58286) 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 -- 172.21.15.49:0/2476605320 learned_addr learned my addr 172.21.15.49:0/2476605320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75de22640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581022b0 0x7fe7581a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75ce20640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 0x7fe75819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 -- 172.21.15.49:0/2476605320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581022b0 msgr2=0x7fe7581a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581022b0 0x7fe7581a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 -- 172.21.15.49:0/2476605320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 msgr2=0x7fe75819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 0x7fe75819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.427+0000 7fe75d621640 1 -- 172.21.15.49:0/2476605320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581a1970 con 0x7fe758100700 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe75d621640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe75400e9c0 tx=0x7fe75400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75400cd80 con 0x7fe758100700 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7540035f0 con 0x7fe758100700 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7581a1c60 con 0x7fe758100700 2024-03-16T08:16:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754010690 con 0x7fe758100700 2024-03-16T08:16:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.428+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7581a2160 con 0x7fe758100700 2024-03-16T08:16:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.429+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7581a2330 con 0x7fe758100700 2024-03-16T08:16:51.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.430+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe7540037a0 con 0x7fe758100700 2024-03-16T08:16:51.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.430+0000 7fe74e7fc640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 0x7fe7440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.430+0000 7fe75ce20640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 0x7fe7440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.430+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe7540ced90 con 0x7fe758100700 2024-03-16T08:16:51.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.431+0000 7fe75ce20640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 0x7fe7440a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe75819b260 tx=0x7fe748005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.435+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe75409be60 con 0x7fe758100700 2024-03-16T08:16:51.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.620+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7581a2500 con 0x7fe758100700 2024-03-16T08:16:51.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.620+0000 7fe74e7fc640 1 -- 172.21.15.49:0/2476605320 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe75409b760 con 0x7fe758100700 2024-03-16T08:16:51.622 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 msgr2=0x7fe7440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 0x7fe7440a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe75819b260 tx=0x7fe748005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 msgr2=0x7fe758199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe75400e9c0 tx=0x7fe75400ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 shutdown_connections 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581022b0 0x7fe7581a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7581018f0 0x7fe75819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe7440a2800 0x7fe7440a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 --2- 172.21.15.49:0/2476605320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758100700 0x7fe758199d50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 >> 172.21.15.49:0/2476605320 conn(0x7fe7580fbc90 msgr2=0x7fe7580fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 shutdown_connections 2024-03-16T08:16:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.622+0000 7fe75f885640 1 -- 172.21.15.49:0/2476605320 wait complete. 2024-03-16T08:16:51.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:51 smithi049 ceph-mon[31942]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:51.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:51 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2112578384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:51 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/212064490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:51 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1856971178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:51.635 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:16:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:51 smithi154 ceph-mon[36471]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:51 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2112578384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:51 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/212064490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:51 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1856971178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- 172.21.15.49:0/4215177509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 msgr2=0x7f8324102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- 172.21.15.49:0/4215177509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324102e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8320009900 tx=0x7f832002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- 172.21.15.49:0/4215177509 shutdown_connections 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- 172.21.15.49:0/4215177509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83241033b0 0x7f8324109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- 172.21.15.49:0/4215177509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324102e70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- 172.21.15.49:0/4215177509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324101800 0x7f8324101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- 172.21.15.49:0/4215177509 >> 172.21.15.49:0/4215177509 conn(0x7f83240fcd70 msgr2=0x7f83240ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- 172.21.15.49:0/4215177509 shutdown_connections 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- 172.21.15.49:0/4215177509 wait complete. 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 Processor -- start 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 -- start start 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324101800 0x7f8324197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.745+0000 7f832bb97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241033b0 0x7f832419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f832410d480 con 0x7f83241029f0 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f832410d300 con 0x7f8324101800 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f832410d600 con 0x7f83241033b0 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58302/0 (socket says 172.21.15.49:58302) 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 -- 172.21.15.49:0/1615680784 learned_addr learned my addr 172.21.15.49:0/1615680784 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329933640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324101800 0x7f8324197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832a134640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241033b0 0x7f832419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 -- 172.21.15.49:0/1615680784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241033b0 msgr2=0x7f832419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241033b0 0x7f832419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 -- 172.21.15.49:0/1615680784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324101800 msgr2=0x7f8324197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324101800 0x7f8324197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 -- 172.21.15.49:0/1615680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832419f720 con 0x7f83241029f0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f8329132640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8320009230 tx=0x7f832002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320041070 con 0x7f83241029f0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832419f9b0 con 0x7f83241029f0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320004450 con 0x7f83241029f0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320003e20 con 0x7f83241029f0 2024-03-16T08:16:51.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.746+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832419fea0 con 0x7f83241029f0 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.747+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8324108b20 con 0x7f83241029f0 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.748+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8320044070 con 0x7f83241029f0 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.749+0000 7f831affd640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 0x7f83100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.749+0000 7f8329933640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 0x7f83100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.749+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f83200f05d0 con 0x7f83241029f0 2024-03-16T08:16:51.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.750+0000 7f8329933640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 0x7f83100a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8324102850 tx=0x7f8314005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.754+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f83200bd050 con 0x7f83241029f0 2024-03-16T08:16:51.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:51 smithi129 ceph-mon[36561]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:16:51.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:51 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2112578384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:51 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/212064490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:51.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:51 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1856971178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:51.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.936+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8324062410 con 0x7f83241029f0 2024-03-16T08:16:51.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.936+0000 7f831affd640 1 -- 172.21.15.49:0/1615680784 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f83200c10f0 con 0x7f83241029f0 2024-03-16T08:16:51.938 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 msgr2=0x7f83100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 0x7f83100a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8324102850 tx=0x7f8314005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 msgr2=0x7f8324198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8320009230 tx=0x7f832002c650 comp rx=0 tx=0).stop 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 shutdown_connections 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241033b0 0x7f832419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f83100a2800 0x7f83100a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241029f0 0x7f8324198040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 --2- 172.21.15.49:0/1615680784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324101800 0x7f8324197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.938+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 >> 172.21.15.49:0/1615680784 conn(0x7f83240fcd70 msgr2=0x7f8324105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.939+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 shutdown_connections 2024-03-16T08:16:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:51.939+0000 7f832bb97640 1 -- 172.21.15.49:0/1615680784 wait complete. 2024-03-16T08:16:51.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.057+0000 7f2c748f7640 1 -- 172.21.15.49:0/2548097291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70102a00 msgr2=0x7f2c70102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.057+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2548097291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70102a00 0x7f2c70102e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c6002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 -- 172.21.15.49:0/2548097291 shutdown_connections 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2548097291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c701033c0 0x7f2c70109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2548097291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70102a00 0x7f2c70102e80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2548097291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c70101810 0x7f2c70101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 -- 172.21.15.49:0/2548097291 >> 172.21.15.49:0/2548097291 conn(0x7f2c700fcdc0 msgr2=0x7f2c700ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 -- 172.21.15.49:0/2548097291 shutdown_connections 2024-03-16T08:16:52.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 -- 172.21.15.49:0/2548097291 wait complete. 2024-03-16T08:16:52.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 Processor -- start 2024-03-16T08:16:52.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.058+0000 7f2c748f7640 1 -- start start 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c70102a00 0x7f2c70198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c701033c0 0x7f2c7019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010d470 con 0x7f2c70101810 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010d2f0 con 0x7f2c701033c0 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010d5f0 con 0x7f2c70102a00 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58314/0 (socket says 172.21.15.49:58314) 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 -- 172.21.15.49:0/2183377564 learned_addr learned my addr 172.21.15.49:0/2183377564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6ed9d640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c701033c0 0x7f2c7019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6dd9b640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c70102a00 0x7f2c70198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 -- 172.21.15.49:0/2183377564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c70102a00 msgr2=0x7f2c70198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c70102a00 0x7f2c70198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 -- 172.21.15.49:0/2183377564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c701033c0 msgr2=0x7f2c7019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c701033c0 0x7f2c7019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 -- 172.21.15.49:0/2183377564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7019f730 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c6e59c640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e9c0 tx=0x7f2c6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400cde0 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7019fa20 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c64010910 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64018a00 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.059+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7019ff40 con 0x7f2c70101810 2024-03-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.061+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c70108b30 con 0x7f2c70101810 2024-03-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.062+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2c6401f020 con 0x7f2c70101810 2024-03-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.062+0000 7f2c5f7fe640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.062+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f2c640cec10 con 0x7f2c70101810 2024-03-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.062+0000 7f2c6dd9b640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.063+0000 7f2c6dd9b640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 0x7f2c580a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2c6002c7b0 tx=0x7f2c60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.067+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2c6409bce0 con 0x7f2c70101810 2024-03-16T08:16:52.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.290+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7002d1b0 con 0x7f2c70101810 2024-03-16T08:16:52.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.290+0000 7f2c5f7fe640 1 -- 172.21.15.49:0/2183377564 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f2c6409b5e0 con 0x7f2c70101810 2024-03-16T08:16:52.292 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:52.292 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:15:07.570394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24491,24505]},"id":1}]} 2024-03-16T08:16:52.292 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 msgr2=0x7f2c580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 0x7f2c580a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2c6002c7b0 tx=0x7f2c60005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 msgr2=0x7f2c70197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e9c0 tx=0x7f2c6400ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 shutdown_connections 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c701033c0 0x7f2c7019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c70102a00 0x7f2c70198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 --2- 172.21.15.49:0/2183377564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c70101810 0x7f2c70197b10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.292+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 >> 172.21.15.49:0/2183377564 conn(0x7f2c700fcdc0 msgr2=0x7f2c70105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.293+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 shutdown_connections 2024-03-16T08:16:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.293+0000 7f2c748f7640 1 -- 172.21.15.49:0/2183377564 wait complete. 2024-03-16T08:16:52.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- 172.21.15.49:0/1476810778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101018f0 msgr2=0x7fce10101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- 172.21.15.49:0/1476810778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101018f0 0x7fce10101d70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce00009900 tx=0x7fce0002bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- 172.21.15.49:0/1476810778 shutdown_connections 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- 172.21.15.49:0/1476810778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101022b0 0x7fce10108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- 172.21.15.49:0/1476810778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101018f0 0x7fce10101d70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- 172.21.15.49:0/1476810778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- 172.21.15.49:0/1476810778 >> 172.21.15.49:0/1476810778 conn(0x7fce100fbc70 msgr2=0x7fce100fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- 172.21.15.49:0/1476810778 shutdown_connections 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- 172.21.15.49:0/1476810778 wait complete. 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 Processor -- start 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 -- start start 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.412+0000 7fce1784d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 0x7fce1019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce1784d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c380 con 0x7fce101018f0 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce1784d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c200 con 0x7fce10100700 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce1784d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c500 con 0x7fce101022b0 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce155e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58334/0 (socket says 172.21.15.49:58334) 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce155e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:44556/0 (socket says 172.21.15.49:44556) 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 -- 172.21.15.49:0/3146002830 learned_addr learned my addr 172.21.15.49:0/3146002830 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce15dea640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 0x7fce1019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 -- 172.21.15.49:0/3146002830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 msgr2=0x7fce1019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 0x7fce1019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 -- 172.21.15.49:0/3146002830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 msgr2=0x7fce10197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 -- 172.21.15.49:0/3146002830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1019f7d0 con 0x7fce101018f0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce15dea640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 0x7fce1019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce14de8640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fce00005ab0 tx=0x7fce0002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00041070 con 0x7fce101018f0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1019fa60 con 0x7fce101018f0 2024-03-16T08:16:52.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00004450 con 0x7fce101018f0 2024-03-16T08:16:52.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00003d70 con 0x7fce101018f0 2024-03-16T08:16:52.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.413+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1019ffa0 con 0x7fce101018f0 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.415+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce101a01d0 con 0x7fce101018f0 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.416+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fce00044070 con 0x7fce101018f0 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.416+0000 7fce067fc640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 0x7fcdfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.416+0000 7fce155e9640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 0x7fcdfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.416+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fce000f07b0 con 0x7fce101018f0 2024-03-16T08:16:52.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.417+0000 7fce155e9640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 0x7fcdfc0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce10101750 tx=0x7fce08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.422+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce000bd880 con 0x7fce101018f0 2024-03-16T08:16:52.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.609+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce101a03a0 con 0x7fce101018f0 2024-03-16T08:16:52.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.610+0000 7fce067fc640 1 -- 172.21.15.49:0/3146002830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fce000bd180 con 0x7fce101018f0 2024-03-16T08:16:52.612 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:52.612 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 msgr2=0x7fcdfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 0x7fcdfc0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce10101750 tx=0x7fce08005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 msgr2=0x7fce101980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fce00005ab0 tx=0x7fce0002c6c0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 shutdown_connections 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce101022b0 0x7fce1019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fcdfc0a2800 0x7fcdfc0a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce101018f0 0x7fce101980f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 --2- 172.21.15.49:0/3146002830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce10100700 0x7fce10197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.612+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 >> 172.21.15.49:0/3146002830 conn(0x7fce100fbc70 msgr2=0x7fce100fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.613+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 shutdown_connections 2024-03-16T08:16:52.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.613+0000 7fce1784d640 1 -- 172.21.15.49:0/3146002830 wait complete. 2024-03-16T08:16:52.624 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:52 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2476605320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.624 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:52 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1615680784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.624 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:52 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2183377564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:52.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T08:15:01.568993+0000', 'modified': '2024-03-16T08:15:07.570394+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24505, 'mds_2': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/272945510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 272945510}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 272945510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/2047612746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 2047612746}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 2047612746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/4234449434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 4234449434}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4234449434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.129:6837/1811728297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 1811728297}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 1811728297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.154:6839/1693946357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 1693946357}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 1693946357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6839/1288779286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 1288779286}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 1288779286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24536, 24491, 24505]} 2024-03-16T08:16:52.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-16T08:16:52.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T08:16:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:52 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2476605320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:52 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1615680784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:52 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2183377564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- 172.21.15.49:0/1621214021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a81022b0 msgr2=0x7f86a8108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 --2- 172.21.15.49:0/1621214021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a81022b0 0x7f86a8108b40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- 172.21.15.49:0/1621214021 shutdown_connections 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 --2- 172.21.15.49:0/1621214021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a81022b0 0x7f86a8108b40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 --2- 172.21.15.49:0/1621214021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 0x7f86a8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 --2- 172.21.15.49:0/1621214021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a8100700 0x7f86a8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- 172.21.15.49:0/1621214021 >> 172.21.15.49:0/1621214021 conn(0x7f86a80fbc90 msgr2=0x7f86a80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- 172.21.15.49:0/1621214021 shutdown_connections 2024-03-16T08:16:52.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- 172.21.15.49:0/1621214021 wait complete. 2024-03-16T08:16:52.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 Processor -- start 2024-03-16T08:16:52.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.734+0000 7f86ae67e640 1 -- start start 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 0x7f86a8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a81022b0 0x7f86a819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810c380 con 0x7f86a8100700 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810c200 con 0x7f86a81018f0 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810c500 con 0x7f86a81022b0 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60060/0 (socket says 172.21.15.49:60060) 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 -- 172.21.15.49:0/2900671743 learned_addr learned my addr 172.21.15.49:0/2900671743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a77fe640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 0x7f86a8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a81022b0 0x7f86a819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 -- 172.21.15.49:0/2900671743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a81022b0 msgr2=0x7f86a819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a81022b0 0x7f86a819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 -- 172.21.15.49:0/2900671743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 msgr2=0x7f86a8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 0x7f86a8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 -- 172.21.15.49:0/2900671743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a819f6d0 con 0x7f86a8100700 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f869c00e9c0 tx=0x7f869c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a57fa640 1 -- 172.21.15.49:0/2900671743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00cd80 con 0x7f86a8100700 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a819f9c0 con 0x7f86a8100700 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a57fa640 1 -- 172.21.15.49:0/2900671743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c0035f0 con 0x7f86a8100700 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.735+0000 7f86a57fa640 1 -- 172.21.15.49:0/2900671743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c010690 con 0x7f86a8100700 2024-03-16T08:16:52.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.736+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a819ff10 con 0x7f86a8100700 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86a57fa640 1 -- 172.21.15.49:0/2900671743 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f869c0037a0 con 0x7f86a8100700 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 Processor -- start 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 -- start start 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e78112fc80 con 0x55e7811cbbd0 2024-03-16T08:16:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7810d8aa0 con 0x55e7811d1730 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86b1261740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e78112bd20 con 0x55e7811cc5c0 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86acc1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86acc1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60076/0 (socket says 172.21.15.49:60076) 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.738+0000 7f86acc1b640 1 -- 172.21.15.49:0/3910921293 learned_addr learned my addr 172.21.15.49:0/3910921293 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86a7fff640 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86a77fe640 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86a57fa640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 0x7f86900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 -- 172.21.15.49:0/3910921293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 msgr2=0x55e7811d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 -- 172.21.15.49:0/3910921293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 msgr2=0x55e7811d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 -- 172.21.15.49:0/3910921293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7810faff0 con 0x55e7811cbbd0 2024-03-16T08:16:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86a57fa640 1 -- 172.21.15.49:0/2900671743 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f869c0ced90 con 0x7f86a8100700 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 0x7f86900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03b6cbbd9d4644e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f868e7fc640 1 -- 172.21.15.49:0/3910921293 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0039040 con 0x55e7811cbbd0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f868e7fc640 1 -- 172.21.15.49:0/3910921293 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a0003e50 con 0x55e7811cbbd0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f868e7fc640 1 -- 172.21.15.49:0/3910921293 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0034440 con 0x55e7811cbbd0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.739+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 0x7f86900a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86a004c000 tx=0x7f86a0045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- 172.21.15.49:0/3910921293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 msgr2=0x55e7811cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a0005f70 comp rx=0 tx=0).stop 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- 172.21.15.49:0/3910921293 shutdown_connections 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 --2- 172.21.15.49:0/3910921293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811cbff0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- 172.21.15.49:0/3910921293 >> 172.21.15.49:0/3910921293 conn(0x55e7811c7140 msgr2=0x55e7811c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- 172.21.15.49:0/3910921293 shutdown_connections 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- 172.21.15.49:0/3910921293 wait complete. 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 -- start start 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.740+0000 7f86b1261740 1 Processor -- start 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e7811d46e0 con 0x55e7811cbbd0 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7810ebef0 con 0x55e7811d1730 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7811d4860 con 0x55e7811cc5c0 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 10 client.0 _ll_register_callbacks cb 0x55e7810e6dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60082/0 (socket says 172.21.15.49:60082) 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 -- 172.21.15.49:0/2931363298 learned_addr learned my addr 172.21.15.49:0/2931363298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a77fe640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 -- 172.21.15.49:0/2931363298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 msgr2=0x55e7811d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 -- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 msgr2=0x55e7811dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 -- 172.21.15.49:0/2931363298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7810f8a30 con 0x55e7811cbbd0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f869c09b6f0 tx=0x7f869c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf1aae81452db99 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86877fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c09be60 con 0x55e7811cbbd0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f869c086070 con 0x55e7811cbbd0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.741+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0003e50 con 0x55e7811cbbd0 2024-03-16T08:16:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f869c09b760 con 0x55e7811cbbd0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 10 client.15210 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e780e442d0 con 0x55e7811cbbd0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a0003e50 con 0x55e7811cbbd0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f869c0afb40 con 0x55e7811cbbd0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 10 client.15210 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 20 client.15210 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e7811e2920 0x55e7811e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 4 client.15210 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e78113d1c0 con 0x55e7811e2920 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.742+0000 7f86a77fe640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e7811e2920 0x55e7811e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.743+0000 7f86a77fe640 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e7811e2920 0x55e7811e3d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24da0fa23d0913ed server_cookie=a4afd53b14f38a2c in_seq=0 out_seq=0 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.743+0000 7f86877fe640 10 client.15210 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86877fe640 1 -- 172.21.15.49:0/2931363298 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e78113d1c0 con 0x55e7811e2920 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86877fe640 10 client.15210 handle_command_reply: tid=0 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 client.15210 shutdown 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 msgr2=0x55e7811d5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f869c09b6f0 tx=0x7f869c0096e0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 shutdown_connections 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e7811e2920 0x55e7811e3d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86877fe640 0 client.15210 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 --2- 172.21.15.49:0/2931363298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d5d90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 >> 172.21.15.49:0/2931363298 conn(0x55e7811c7140 msgr2=0x55e7811c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 shutdown_connections 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 1 -- 172.21.15.49:0/2931363298 wait complete. 2024-03-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.744+0000 7f86b1261740 20 client.15210 trim_cache size 0 max 16384 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 Processor -- start 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 -- start start 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e20d0 con 0x55e7811cbbd0 2024-03-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e27a0 con 0x55e7811d1730 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86b1261740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e2e70 con 0x55e7811cc5c0 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60092/0 (socket says 172.21.15.49:60092) 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.762+0000 7f86a7fff640 1 -- 172.21.15.49:0/1503092491 learned_addr learned my addr 172.21.15.49:0/1503092491 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a77fe640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86acc1b640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 -- 172.21.15.49:0/1503092491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 msgr2=0x55e7811dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 -- 172.21.15.49:0/1503092491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 msgr2=0x55e7811d5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 -- 172.21.15.49:0/1503092491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e780f0e6b0 con 0x55e7811cbbd0 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86acc1b640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f86a7fff640 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c6a0 tx=0x7f869c099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5085442179a99c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f868dffb640 1 -- 172.21.15.49:0/1503092491 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c09f8b0 con 0x55e7811cbbd0 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f868dffb640 1 -- 172.21.15.49:0/1503092491 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c0a3f00 con 0x55e7811cbbd0 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.763+0000 7f868dffb640 1 -- 172.21.15.49:0/1503092491 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0a3cd0 con 0x55e7811cbbd0 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- 172.21.15.49:0/1503092491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 msgr2=0x55e7811d59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c6a0 tx=0x7f869c099c70 comp rx=0 tx=0).stop 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- 172.21.15.49:0/1503092491 shutdown_connections 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811d1730 0x55e7811d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- 172.21.15.49:0/1503092491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811cbbd0 0x55e7811d59a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- 172.21.15.49:0/1503092491 >> 172.21.15.49:0/1503092491 conn(0x55e7811c7140 msgr2=0x55e7811e16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- 172.21.15.49:0/1503092491 shutdown_connections 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- 172.21.15.49:0/1503092491 wait complete. 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- start start 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 Processor -- start 2024-03-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811cbbd0 0x55e7811d8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e3c10 con 0x55e7811d1730 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e3540 con 0x55e7811cbbd0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7811e42e0 con 0x55e7811cc5c0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86b1261740 10 client.0 _ll_register_callbacks cb 0x7f869c003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86acc1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86acc1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60104/0 (socket says 172.21.15.49:60104) 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811cbbd0 0x55e7811d8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.764+0000 7f86acc1b640 1 -- 172.21.15.49:0/2947454747 learned_addr learned my addr 172.21.15.49:0/2947454747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86a77fe640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 -- 172.21.15.49:0/2947454747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 msgr2=0x55e7811d9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 -- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811cbbd0 msgr2=0x55e7811d8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811cbbd0 0x55e7811d8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 -- 172.21.15.49:0/2947454747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7811dc5d0 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86acc1b640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f86a00325d0 tx=0x7f86a0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7b752d14a9b913 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86877fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0039040 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f86a0003b70 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869c0a3f00 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f86a0034440 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86b1261740 10 client.15216 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e781248750 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.765+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c0a3f00 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f86a003f040 con 0x55e7811d1730 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 10 client.15216 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 20 client.15216 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e78124cee0 0x55e78124bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 4 client.15216 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "status"}) -- 0x55e781137510 con 0x55e78124cee0 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e78124cee0 0x55e78124bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86877fe640 10 client.15216 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86a7fff640 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e78124cee0 0x55e78124bbf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7809b73b2b942d13 server_cookie=c8cb481710a9fa1a in_seq=0 out_seq=0 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86877fe640 1 -- 172.21.15.49:0/2947454747 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e781137510 con 0x55e78124cee0 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86877fe640 10 client.15216 handle_command_reply: tid=0 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.766+0000 7f86b1261740 1 client.15216 shutdown 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 msgr2=0x55e7811dbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f86a00325d0 tx=0x7f86a0005ef0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 shutdown_connections 2024-03-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e78124cee0 0x55e78124bbf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7811d1730 0x55e7811dbfe0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7811cc5c0 0x55e7811d9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86877fe640 0 client.15216 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 --2- 172.21.15.49:0/2947454747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7811cbbd0 0x55e7811d8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 >> 172.21.15.49:0/2947454747 conn(0x55e7811c7140 msgr2=0x55e7811d5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 shutdown_connections 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 1 -- 172.21.15.49:0/2947454747 wait complete. 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86b1261740 20 client.15216 trim_cache size 0 max 16384 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:52.775 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24491, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 106.202644998, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 126.881748704 2024-03-16T08:16:52.776 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 msgr2=0x7f86900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 0x7f86900a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86a004c000 tx=0x7f86a0045000 comp rx=0 tx=0).stop 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 msgr2=0x7f86a8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f869c00e9c0 tx=0x7f869c00ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 shutdown_connections 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a81022b0 0x7f86a819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86a81018f0 0x7f86a8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f86900a2800 0x7f86900a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 --2- 172.21.15.49:0/2900671743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a8100700 0x7f86a8197ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 >> 172.21.15.49:0/2900671743 conn(0x7f86a80fbc90 msgr2=0x7f86a80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 shutdown_connections 2024-03-16T08:16:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.767+0000 7f86ae67e640 1 -- 172.21.15.49:0/2900671743 wait complete. 2024-03-16T08:16:52.779 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T08:16:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:52 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2476605320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:52 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1615680784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:16:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:52 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2183377564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:16:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 -- 172.21.15.49:0/4035671491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8701018f0 msgr2=0x7ff870101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 --2- 172.21.15.49:0/4035671491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8701018f0 0x7ff870101d70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009900 tx=0x7ff85c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 -- 172.21.15.49:0/4035671491 shutdown_connections 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 --2- 172.21.15.49:0/4035671491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701022b0 0x7ff870108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 --2- 172.21.15.49:0/4035671491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8701018f0 0x7ff870101d70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 --2- 172.21.15.49:0/4035671491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff870100700 0x7ff870100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 -- 172.21.15.49:0/4035671491 >> 172.21.15.49:0/4035671491 conn(0x7ff8700fbc90 msgr2=0x7ff8700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 -- 172.21.15.49:0/4035671491 shutdown_connections 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.887+0000 7ff87713b640 1 -- 172.21.15.49:0/4035671491 wait complete. 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 Processor -- start 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 -- start start 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 0x7ff8701096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 0x7ff870109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff87010d4a0 con 0x7ff870100700 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff87010d320 con 0x7ff8701018f0 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff87713b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff87010d620 con 0x7ff8701022b0 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff874ed7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff874ed7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60128/0 (socket says 172.21.15.49:60128) 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 0x7ff8701096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff874ed7640 1 -- 172.21.15.49:0/4120391046 learned_addr learned my addr 172.21.15.49:0/4120391046 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.888+0000 7ff8756d8640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 0x7ff870109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 -- 172.21.15.49:0/4120391046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 msgr2=0x7ff870109c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 0x7ff870109c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 -- 172.21.15.49:0/4120391046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 msgr2=0x7ff8701096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 0x7ff8701096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 -- 172.21.15.49:0/4120391046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87010a490 con 0x7ff870100700 2024-03-16T08:16:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff8756d8640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 0x7ff870109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff86ffff640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 0x7ff8701096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff874ed7640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff86dffb640 1 -- 172.21.15.49:0/4120391046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400cd80 con 0x7ff870100700 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff86dffb640 1 -- 172.21.15.49:0/4120391046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8640035f0 con 0x7ff870100700 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8701a1c20 con 0x7ff870100700 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff86dffb640 1 -- 172.21.15.49:0/4120391046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864010690 con 0x7ff870100700 2024-03-16T08:16:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.889+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8701a2190 con 0x7ff870100700 2024-03-16T08:16:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.891+0000 7ff86dffb640 1 -- 172.21.15.49:0/4120391046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff8640037a0 con 0x7ff870100700 2024-03-16T08:16:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 Processor -- start 2024-03-16T08:16:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 -- start start 2024-03-16T08:16:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff86dffb640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 0x7ff8600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417e0730 0x5635417e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56354173ec80 con 0x5635417dabd0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635416e7aa0 con 0x5635417e0730 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff879e08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56354173ad20 con 0x5635417db5c0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60134/0 (socket says 172.21.15.49:60134) 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 -- 172.21.15.49:0/500065531 learned_addr learned my addr 172.21.15.49:0/500065531 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff874ed7640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 0x7ff8600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff86ffff640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417e0730 0x5635417e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff86dffb640 1 -- 172.21.15.49:0/4120391046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff8640ceda0 con 0x7ff870100700 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 -- 172.21.15.49:0/500065531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 msgr2=0x5635417e00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 -- 172.21.15.49:0/500065531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417e0730 msgr2=0x5635417e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417e0730 0x5635417e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 -- 172.21.15.49:0/500065531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563541709ff0 con 0x5635417dabd0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff86ffff640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff874ed7640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 0x7ff8600a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff864097bd0 tx=0x7ff86401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.892+0000 7ff8756d8640 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff868013f30 tx=0x7ff868009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ef1527d6eff1d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff85affd640 1 -- 172.21.15.49:0/500065531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86800fd10 con 0x5635417dabd0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff85affd640 1 -- 172.21.15.49:0/500065531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff868013130 con 0x5635417dabd0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff85affd640 1 -- 172.21.15.49:0/500065531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868004290 con 0x5635417dabd0 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 -- 172.21.15.49:0/500065531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 msgr2=0x5635417daff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff868013f30 tx=0x7ff868009ee0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 -- 172.21.15.49:0/500065531 shutdown_connections 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417e0730 0x5635417e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 --2- 172.21.15.49:0/500065531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417dabd0 0x5635417daff0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 -- 172.21.15.49:0/500065531 >> 172.21.15.49:0/500065531 conn(0x5635417d6140 msgr2=0x5635417d6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 -- 172.21.15.49:0/500065531 shutdown_connections 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.893+0000 7ff879e08740 1 -- 172.21.15.49:0/500065531 wait complete. 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 -- start start 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 Processor -- start 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5635417e3680 con 0x5635417e0730 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635416faef0 con 0x5635417dabd0 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5635417e3800 con 0x5635417db5c0 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff879e08740 10 client.0 _ll_register_callbacks cb 0x5635416f5dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff8756d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff8756d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60136/0 (socket says 172.21.15.49:60136) 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:53098/0 (socket says 172.21.15.49:53098) 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff8756d8640 1 -- 172.21.15.49:0/2440085700 learned_addr learned my addr 172.21.15.49:0/2440085700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.894+0000 7ff8756d8640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 -- 172.21.15.49:0/2440085700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 msgr2=0x5635417e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 -- 172.21.15.49:0/2440085700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 msgr2=0x5635417ec2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 -- 172.21.15.49:0/2440085700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563541707a30 con 0x5635417db5c0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff8756d8640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff86ffff640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009230 tx=0x7ff85c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139c5f84b014bf91 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:16:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c005d50 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff85c0046c0 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff868013130 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff85c04d070 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff879e08740 10 client.25111 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5635414532d0 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff868013130 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff85c003a30 con 0x5635417db5c0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.895+0000 7ff879e08740 10 client.25111 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff879e08740 10 client.25111 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff879e08740 20 client.25111 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5635417f17b0 0x5635417f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff879e08740 4 client.25111 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56354174c1c0 con 0x5635417f17b0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff8756d8640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5635417f17b0 0x5635417f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff8756d8640 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5635417f17b0 0x5635417f2c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3671b6266d1515c6 server_cookie=48daecc696735635 in_seq=0 out_seq=0 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.896+0000 7ff84ffff640 10 client.25111 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff84ffff640 1 -- 172.21.15.49:0/2440085700 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56354174c1c0 con 0x5635417f17b0 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff84ffff640 10 client.25111 handle_command_reply: tid=0 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 client.25111 shutdown 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 msgr2=0x5635417e52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009230 tx=0x7ff85c005270 comp rx=0 tx=0).stop 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 shutdown_connections 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5635417f17b0 0x5635417f2c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417e0730 0x5635417ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff84ffff640 0 client.25111 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417db5c0 0x5635417e52d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 --2- 172.21.15.49:0/2440085700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 >> 172.21.15.49:0/2440085700 conn(0x5635417d6140 msgr2=0x5635417d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 shutdown_connections 2024-03-16T08:16:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 1 -- 172.21.15.49:0/2440085700 wait complete. 2024-03-16T08:16:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.897+0000 7ff879e08740 20 client.25111 trim_cache size 0 max 16384 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.915+0000 7ff879e08740 1 Processor -- start 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.915+0000 7ff879e08740 1 -- start start 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.915+0000 7ff879e08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417d6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.915+0000 7ff879e08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.915+0000 7ff879e08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff879e08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5635417f0f60 con 0x5635417db5c0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff879e08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635417f1630 con 0x5635417dabd0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff879e08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5635417f1d00 con 0x5635417e0730 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60144/0 (socket says 172.21.15.49:60144) 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 -- 172.21.15.49:0/997998626 learned_addr learned my addr 172.21.15.49:0/997998626 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff8756d8640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417d6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff8756d8640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 -- 172.21.15.49:0/997998626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 msgr2=0x5635417dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 -- 172.21.15.49:0/997998626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 msgr2=0x5635417d6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417d6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 -- 172.21.15.49:0/997998626 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354151d6b0 con 0x5635417db5c0 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff8756d8640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:52.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff86ffff640 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02cfd0 tx=0x7ff85c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88eb23a1e7368ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff85a7fc640 1 -- 172.21.15.49:0/997998626 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c005d50 con 0x5635417db5c0 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff85a7fc640 1 -- 172.21.15.49:0/997998626 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c041070 con 0x5635417db5c0 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.916+0000 7ff85a7fc640 1 -- 172.21.15.49:0/997998626 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0035f0 con 0x5635417db5c0 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- 172.21.15.49:0/997998626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 msgr2=0x5635417ec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02cfd0 tx=0x7ff85c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- 172.21.15.49:0/997998626 shutdown_connections 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417ec260 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 --2- 172.21.15.49:0/997998626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417d6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- 172.21.15.49:0/997998626 >> 172.21.15.49:0/997998626 conn(0x5635417d6140 msgr2=0x5635417d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- 172.21.15.49:0/997998626 shutdown_connections 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- 172.21.15.49:0/997998626 wait complete. 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 -- start start 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.917+0000 7ff879e08740 1 Processor -- start 2024-03-16T08:16:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5635417f2aa0 con 0x5635417db5c0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635417f23d0 con 0x5635417dabd0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5635417f3170 con 0x5635417e0730 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 10 client.0 _ll_register_callbacks cb 0x7ff85c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60158/0 (socket says 172.21.15.49:60158) 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 -- 172.21.15.49:0/1750840271 learned_addr learned my addr 172.21.15.49:0/1750840271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff8756d8640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff8756d8640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 -- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 msgr2=0x5635417eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417eb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 -- 172.21.15.49:0/1750840271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 msgr2=0x5635417e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 -- 172.21.15.49:0/1750840271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635417eb8b0 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff8756d8640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff86ffff640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02c170 tx=0x7ff85c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ea8d1e14067c12 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff84ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c005d50 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff85c046ab0 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c041070 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff85c06c050 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.918+0000 7ff879e08740 10 client.15234 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5635418578a0 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c041070 con 0x5635417db5c0 2024-03-16T08:16:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff85c004750 con 0x5635417db5c0 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 10 client.15234 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 20 client.15234 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56354185c120 0x56354185ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 4 client.15234 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "status"}) -- 0x563541746510 con 0x56354185c120 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff8756d8640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56354185c120 0x56354185ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff84ffff640 10 client.15234 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.919+0000 7ff8756d8640 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56354185c120 0x56354185ae30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1ae432e08c2b8be server_cookie=c309a8e5523b59be in_seq=0 out_seq=0 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff84ffff640 1 -- 172.21.15.49:0/1750840271 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563541746510 con 0x56354185c120 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff84ffff640 10 client.15234 handle_command_reply: tid=0 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 client.15234 shutdown 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 msgr2=0x5635417e8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02c170 tx=0x7ff85c009520 comp rx=0 tx=0).stop 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 shutdown_connections 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56354185c120 0x56354185ae30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5635417e0730 0x5635417eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff84ffff640 0 client.15234 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635417db5c0 0x5635417e8930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 --2- 172.21.15.49:0/1750840271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635417dabd0 0x5635417e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 >> 172.21.15.49:0/1750840271 conn(0x5635417d6140 msgr2=0x5635417dfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 shutdown_connections 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 1 -- 172.21.15.49:0/1750840271 wait complete. 2024-03-16T08:16:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.920+0000 7ff879e08740 20 client.15234 trim_cache size 0 max 16384 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24505, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 107.364340678, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 124.29514641599999 2024-03-16T08:16:52.929 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 msgr2=0x7ff8600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 0x7ff8600a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff864097bd0 tx=0x7ff86401d000 comp rx=0 tx=0).stop 2024-03-16T08:16:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 msgr2=0x7ff87010b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 shutdown_connections 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8701022b0 0x7ff870109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8701018f0 0x7ff8701096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff8600a2880 0x7ff8600a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 --2- 172.21.15.49:0/4120391046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff870100700 0x7ff87010b080 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 >> 172.21.15.49:0/4120391046 conn(0x7ff8700fbc90 msgr2=0x7ff870104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 shutdown_connections 2024-03-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:52.921+0000 7ff87713b640 1 -- 172.21.15.49:0/4120391046 wait complete. 2024-03-16T08:16:52.933 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T08:16:53.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 -- 172.21.15.49:0/587729300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a41018d0 msgr2=0x7f87a4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 --2- 172.21.15.49:0/587729300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a41018d0 0x7f87a4101d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8798009860 tx=0x7f879802bb70 comp rx=0 tx=0).stop 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 -- 172.21.15.49:0/587729300 shutdown_connections 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 --2- 172.21.15.49:0/587729300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a4102290 0x7f87a4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 --2- 172.21.15.49:0/587729300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a41018d0 0x7f87a4101d50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 --2- 172.21.15.49:0/587729300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 0x7f87a4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.040+0000 7f87a971b640 1 -- 172.21.15.49:0/587729300 >> 172.21.15.49:0/587729300 conn(0x7f87a40fbc70 msgr2=0x7f87a40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- 172.21.15.49:0/587729300 shutdown_connections 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- 172.21.15.49:0/587729300 wait complete. 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 Processor -- start 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- start start 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 0x7f87a4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a4102290 0x7f87a419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410c340 con 0x7f87a41018d0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410c1c0 con 0x7f87a41006e0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.041+0000 7f87a971b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410c4c0 con 0x7f87a4102290 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60180/0 (socket says 172.21.15.49:60180) 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 -- 172.21.15.49:0/3245893261 learned_addr learned my addr 172.21.15.49:0/3245893261 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a37fe640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a4102290 0x7f87a419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 0x7f87a4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 -- 172.21.15.49:0/3245893261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a4102290 msgr2=0x7f87a419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a4102290 0x7f87a419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 -- 172.21.15.49:0/3245893261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 msgr2=0x7f87a4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 0x7f87a4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 -- 172.21.15.49:0/3245893261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a419f740 con 0x7f87a41018d0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8798005940 tx=0x7f8798032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f878bfff640 1 -- 172.21.15.49:0/3245893261 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798041020 con 0x7f87a41018d0 2024-03-16T08:16:53.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f878bfff640 1 -- 172.21.15.49:0/3245893261 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879802c9f0 con 0x7f87a41018d0 2024-03-16T08:16:53.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a419f9d0 con 0x7f87a41018d0 2024-03-16T08:16:53.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f878bfff640 1 -- 172.21.15.49:0/3245893261 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798039040 con 0x7f87a41018d0 2024-03-16T08:16:53.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.042+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a419fec0 con 0x7f87a41018d0 2024-03-16T08:16:53.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.044+0000 7f878bfff640 1 -- 172.21.15.49:0/3245893261 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8798044020 con 0x7f87a41018d0 2024-03-16T08:16:53.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 Processor -- start 2024-03-16T08:16:53.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 -- start start 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e4291ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4297730 0x55a4e4299bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4e41f5c80 con 0x55a4e42925c0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4e419eaa0 con 0x55a4e4291bd0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4e41f1d20 con 0x55a4e4297730 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f878bfff640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 0x7f878c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60194/0 (socket says 172.21.15.49:60194) 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4297730 0x55a4e4299bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a2ffd640 1 -- 172.21.15.49:0/2105097034 learned_addr learned my addr 172.21.15.49:0/2105097034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a37fe640 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e4291ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f87a37fe640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 0x7f878c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.045+0000 7f878bfff640 1 -- 172.21.15.49:0/3245893261 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f87980f0380 con 0x7f87a41018d0 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 -- 172.21.15.49:0/2105097034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4297730 msgr2=0x55a4e4299bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4297730 0x55a4e4299bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 -- 172.21.15.49:0/2105097034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 msgr2=0x55a4e42970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 -- 172.21.15.49:0/2105097034 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e41c0ff0 con 0x55a4e4291bd0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 0x7f878c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f879400cd00 tx=0x7f8794011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87a37fe640 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e4291ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f879400ef50 tx=0x7f879400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1135d2ec847e512b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87897fa640 1 -- 172.21.15.49:0/2105097034 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794021070 con 0x55a4e4291bd0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87897fa640 1 -- 172.21.15.49:0/2105097034 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879401cd00 con 0x55a4e4291bd0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.046+0000 7f87897fa640 1 -- 172.21.15.49:0/2105097034 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794004230 con 0x55a4e4291bd0 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- 172.21.15.49:0/2105097034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 msgr2=0x55a4e4291ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e4291ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f879400ef50 tx=0x7f879400c800 comp rx=0 tx=0).stop 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- 172.21.15.49:0/2105097034 shutdown_connections 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4297730 0x55a4e4299bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e42970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/2105097034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e4291ff0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- 172.21.15.49:0/2105097034 >> 172.21.15.49:0/2105097034 conn(0x55a4e428d140 msgr2=0x55a4e428d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- 172.21.15.49:0/2105097034 shutdown_connections 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- 172.21.15.49:0/2105097034 wait complete. 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- start start 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 Processor -- start 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e429c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4e429a620 con 0x55a4e4291bd0 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4e41b1ef0 con 0x55a4e4297730 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4e429a7a0 con 0x55a4e42925c0 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87ac3e2740 10 client.0 _ll_register_callbacks cb 0x55a4e41acdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.047+0000 7f87a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e429c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60196/0 (socket says 172.21.15.49:60196) 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 -- 172.21.15.49:0/3067232965 learned_addr learned my addr 172.21.15.49:0/3067232965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a37fe640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 -- 172.21.15.49:0/3067232965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 msgr2=0x55a4e429c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e429c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 -- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 msgr2=0x55a4e42a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 -- 172.21.15.49:0/3067232965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e41bea30 con 0x55a4e4291bd0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e429c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a37fe640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f879c012700 tx=0x7f879c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958ee7bb5be653a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f8786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c004030 con 0x55a4e4291bd0 2024-03-16T08:16:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f879c0041d0 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879401cd00 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f879c023070 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87ac3e2740 10 client.15246 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.048+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a4e3f0a2d0 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879401cd00 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f879c003980 con 0x55a4e4291bd0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 10 client.15246 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 20 client.15246 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e42a8840 0x55a4e42a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 4 client.15246 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4e42031c0 con 0x55a4e42a8840 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e42a8840 0x55a4e42a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f8786ff5640 10 client.15246 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.049+0000 7f87a27fc640 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e42a8840 0x55a4e42a9cb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebfb1e41b72d1c0 server_cookie=ce2c12e1a66a3da7 in_seq=0 out_seq=0 2024-03-16T08:16:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f8786ff5640 1 -- 172.21.15.49:0/3067232965 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a4e42031c0 con 0x55a4e42a8840 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f8786ff5640 10 client.15246 handle_command_reply: tid=0 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 client.15246 shutdown 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 msgr2=0x55a4e429bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f879c012700 tx=0x7f879c012bc0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 shutdown_connections 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e42a8840 0x55a4e42a9cb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e429c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f8786ff5640 0 client.15246 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/3067232965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4291bd0 0x55a4e429bd50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 >> 172.21.15.49:0/3067232965 conn(0x55a4e428d140 msgr2=0x55a4e428c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 shutdown_connections 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 1 -- 172.21.15.49:0/3067232965 wait complete. 2024-03-16T08:16:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.050+0000 7f87ac3e2740 20 client.15246 trim_cache size 0 max 16384 2024-03-16T08:16:53.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 Processor -- start 2024-03-16T08:16:53.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 -- start start 2024-03-16T08:16:53.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e42a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e42a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42a7ff0 con 0x55a4e4297730 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42a86c0 con 0x55a4e4291bd0 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.068+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42a8d90 con 0x55a4e42925c0 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60198/0 (socket says 172.21.15.49:60198) 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 -- 172.21.15.49:0/1072101389 learned_addr learned my addr 172.21.15.49:0/1072101389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e42a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e42a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 -- 172.21.15.49:0/1072101389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 msgr2=0x55a4e42a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e42a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 -- 172.21.15.49:0/1072101389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 msgr2=0x55a4e42a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e42a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 -- 172.21.15.49:0/1072101389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e3fd46b0 con 0x55a4e4297730 2024-03-16T08:16:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f87a37fe640 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8794003a50 tx=0x7f8794003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377934530ee2cbf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f8788ff9640 1 -- 172.21.15.49:0/1072101389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794021040 con 0x55a4e4297730 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f8788ff9640 1 -- 172.21.15.49:0/1072101389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8794034070 con 0x55a4e4297730 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.069+0000 7f8788ff9640 1 -- 172.21.15.49:0/1072101389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794005070 con 0x55a4e4297730 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1072101389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 msgr2=0x55a4e4290ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8794003a50 tx=0x7f8794003b30 comp rx=0 tx=0).stop 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1072101389 shutdown_connections 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e4297730 0x55a4e4290ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e42925c0 0x55a4e42a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1072101389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4291bd0 0x55a4e42a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1072101389 >> 172.21.15.49:0/1072101389 conn(0x55a4e428d140 msgr2=0x55a4e42903a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1072101389 shutdown_connections 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1072101389 wait complete. 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- start start 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 Processor -- start 2024-03-16T08:16:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 0x55a4e429e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42a9b30 con 0x55a4e42925c0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42a9460 con 0x55a4e4297730 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4e42aa200 con 0x55a4e4291bd0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87ac3e2740 10 client.0 _ll_register_callbacks cb 0x7f879c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87a2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 0x55a4e429e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60208/0 (socket says 172.21.15.49:60208) 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.070+0000 7f87a27fc640 1 -- 172.21.15.49:0/1370563726 learned_addr learned my addr 172.21.15.49:0/1370563726 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a37fe640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 -- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 msgr2=0x55a4e42a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 -- 172.21.15.49:0/1370563726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 msgr2=0x55a4e429eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 -- 172.21.15.49:0/1370563726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e42a2030 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a27fc640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87a2ffd640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 0x55a4e429e5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f879c003df0 tx=0x7f879c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad7e0355a22df36 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f8786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0050c0 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f879c005260 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8794034070 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f879c005480 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87ac3e2740 10 client.25135 fetch_fsmap learned FSMap version 23 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a4e42a33a0 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.071+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8794034070 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f879c028480 con 0x55a4e4291bd0 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 10 client.25135 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 10 client.25135 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 20 client.25135 populate_metadata read hostname 'smithi049' 2024-03-16T08:16:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e4312d20 0x55a4e4311a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 4 client.25135 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "status"}) -- 0x55a4e41fd510 con 0x55a4e4312d20 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87a27fc640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e4312d20 0x55a4e4311a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f8786ff5640 10 client.25135 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87a27fc640 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e4312d20 0x55a4e4311a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663a6becf21f2a9d server_cookie=ae9d4af6f6d6d0b3 in_seq=0 out_seq=0 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f8786ff5640 1 -- 172.21.15.49:0/1370563726 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a4e41fd510 con 0x55a4e4312d20 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f8786ff5640 10 client.25135 handle_command_reply: tid=0 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.072+0000 7f87ac3e2740 1 client.25135 shutdown 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 msgr2=0x55a4e429e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 0x55a4e429e5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f879c003df0 tx=0x7f879c0038a0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 shutdown_connections 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4e4312d20 0x55a4e4311a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4e4297730 0x55a4e42a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f8786ff5640 0 client.25135 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4e42925c0 0x55a4e429eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 --2- 172.21.15.49:0/1370563726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4e4291bd0 0x55a4e429e5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 >> 172.21.15.49:0/1370563726 conn(0x55a4e428d140 msgr2=0x55a4e42903a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 shutdown_connections 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 1 -- 172.21.15.49:0/1370563726 wait complete. 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87ac3e2740 20 client.25135 trim_cache size 0 max 16384 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "053f2308-e36c-11ee-95c9-87774f69a715", 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24536, 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-16T08:16:53.080 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 110.541173934, 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 116.066683497 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:16:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 msgr2=0x7f878c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 0x7f878c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f879400cd00 tx=0x7f8794011000 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 msgr2=0x7f87a4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8798005940 tx=0x7f8798032470 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 shutdown_connections 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a4102290 0x7f87a419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f878c0a2800 0x7f878c0a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a41018d0 0x7f87a4198060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 --2- 172.21.15.49:0/3245893261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87a41006e0 0x7f87a4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 >> 172.21.15.49:0/3245893261 conn(0x7f87a40fbc70 msgr2=0x7f87a40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 shutdown_connections 2024-03-16T08:16:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:53.073+0000 7f87a971b640 1 -- 172.21.15.49:0/3245893261 wait complete. 2024-03-16T08:16:53.085 INFO:teuthology.run_tasks:Running task kclient... 2024-03-16T08:16:53.108 INFO:tasks.kclient:Mounting kernel clients... 2024-03-16T08:16:53.108 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-16T08:16:53.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:53.108 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-16T08:16:53.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:53.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-16T08:16:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-03-16T08:16:53.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:16:53.198 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-16T08:16:53.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T08:16:53.198 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T08:16:53.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T08:16:53.492 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:16:53.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:53 smithi049 ceph-mon[31942]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:53.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:53 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3146002830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:53.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:53 smithi154 ceph-mon[36471]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:53.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:53 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3146002830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:53.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:53 smithi129 ceph-mon[36561]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:16:53.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:53 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3146002830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 -- 172.21.15.49:0/1134993274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e41075e0 msgr2=0x7f88e410f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/1134993274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e41075e0 0x7f88e410f190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009990 tx=0x7f88d4009ca0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 -- 172.21.15.49:0/1134993274 shutdown_connections 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/1134993274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e41075e0 0x7f88e410f190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/1134993274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406aa70 0x7f88e406ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/1134993274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e406b420 0x7f88e4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 -- 172.21.15.49:0/1134993274 >> 172.21.15.49:0/1134993274 conn(0x7f88e4077220 msgr2=0x7f88e4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:55.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 -- 172.21.15.49:0/1134993274 shutdown_connections 2024-03-16T08:16:55.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.192+0000 7f88e8eaf700 1 -- 172.21.15.49:0/1134993274 wait complete. 2024-03-16T08:16:55.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 Processor -- start 2024-03-16T08:16:55.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 -- start start 2024-03-16T08:16:55.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 0x7f88e41a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e41075e0 0x7f88e419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.193+0000 7f88e8eaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41125f0 con 0x7f88e406aa70 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e8eaf700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4112470 con 0x7f88e406b420 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e8eaf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4112770 con 0x7f88e41075e0 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60232/0 (socket says 172.21.15.49:60232) 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 -- 172.21.15.49:0/10391159 learned_addr learned my addr 172.21.15.49:0/10391159 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e1d9b700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 0x7f88e41a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e2d9d700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e41075e0 0x7f88e419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 -- 172.21.15.49:0/10391159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e41075e0 msgr2=0x7f88e419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e41075e0 0x7f88e419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 -- 172.21.15.49:0/10391159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 msgr2=0x7f88e41a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 0x7f88e41a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 -- 172.21.15.49:0/10391159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e419b150 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e1d9b700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 0x7f88e41a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e259c700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00ef10 tx=0x7f88dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00c9c0 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e419b3e0 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88dc019070 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc007e60 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.194+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a7700 con 0x7f88e406aa70 2024-03-16T08:16:55.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.195+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e4068f90 con 0x7f88e406aa70 2024-03-16T08:16:55.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.197+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f88dc012050 con 0x7f88e406aa70 2024-03-16T08:16:55.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.197+0000 7f88db7fe700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 0x7f88d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:55.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.197+0000 7f88e1d9b700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 0x7f88d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:55.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.197+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f88dc0ce9d0 con 0x7f88e406aa70 2024-03-16T08:16:55.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.198+0000 7f88e1d9b700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 0x7f88d00a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88e41088f0 tx=0x7f88cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:55.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.202+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f88dc010040 con 0x7f88e406aa70 2024-03-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.458+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88e41a78d0 con 0x7f88e406aa70 2024-03-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.458+0000 7f88db7fe700 1 -- 172.21.15.49:0/10391159 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f88dc09baa0 con 0x7f88e406aa70 2024-03-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 msgr2=0x7f88d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 0x7f88d00a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88e41088f0 tx=0x7f88cc009260 comp rx=0 tx=0).stop 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 msgr2=0x7f88e41a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00ef10 tx=0x7f88dc00c3a0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 shutdown_connections 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f88d00a2890 0x7f88d00a4d50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e41075e0 0x7f88e419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.460+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88e406aa70 0x7f88e41a0830 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.461+0000 7f88e8eaf700 1 --2- 172.21.15.49:0/10391159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e406b420 0x7f88e41a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:55.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.461+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 >> 172.21.15.49:0/10391159 conn(0x7f88e4077220 msgr2=0x7f88e410d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:55.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.461+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 shutdown_connections 2024-03-16T08:16:55.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:55.461+0000 7f88e8eaf700 1 -- 172.21.15.49:0/10391159 wait complete. 2024-03-16T08:16:55.563 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:55 smithi049 ceph-mon[31942]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:55.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:55 smithi154 ceph-mon[36471]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:55.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:55 smithi129 ceph-mon[36561]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T08:16:56.211 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T08:16:56.212 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T08:16:56.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:56.212 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-03-16T08:16:56.228 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-03-16T08:16:56.229 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42093sec preferred_lft 42093sec 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:16:56.230 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-03-16T08:16:56.230 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T08:16:56.231 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-16T08:16:56.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:56.444 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:56 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/10391159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:16:56.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:56.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:56.456 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T08:16:56.488 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-03-16T08:16:56.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:56.489 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-03-16T08:16:56.564 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-03-16T08:16:56.565 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T08:16:56.565 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T08:16:56.565 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T08:16:56.565 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T08:16:56.565 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:16:56.565 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-16T08:16:56.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T08:16:56.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T08:16:56.566 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T08:16:56.566 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-16T08:16:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:56 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/10391159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:16:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:56.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:56.859 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-16T08:16:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:56 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/10391159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:16:56.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:56.875 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-03-16T08:16:56.930 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:16:56.930 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-16T08:16:56.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T08:16:56.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T08:16:56.930 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-16T08:16:57.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:57.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.060 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T08:16:57.061 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-16T08:16:57.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:57.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.222 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:16:57.223 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-16T08:16:57.223 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-03-16T08:16:57.223 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T08:16:57.223 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-16T08:16:57.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:57.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:16:57.313 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T08:16:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T08:16:57.329 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:16:57.329 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T08:16:57.382 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-16T08:16:57.382 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-16T08:16:57.382 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/f9081ef2-12dc-45ae-ba21-10abb3f5d3ca 2024-03-16T08:16:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:57.382 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/f9081ef2-12dc-45ae-ba21-10abb3f5d3ca /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T08:16:57.471 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T08:16:57.471 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-16T08:16:57.471 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-03-16T08:16:57.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:57 smithi049 ceph-mon[31942]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T08:16:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-16T08:16:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-03-16T08:16:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.129:6789,172.21.15.154:6789" 2024-03-16T08:16:57.577 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.129:6789,172.21.15.154:6789:/volumes/qa/sv_1/f9081ef2-12dc-45ae-ba21-10abb3f5d3ca 2024-03-16T08:16:57.577 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=053f2308-e36c-11ee-95c9-87774f69a715" will pass to kernel 2024-03-16T08:16:57.577 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-16T08:16:57.577 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-03-16T08:16:57.577 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T08:16:57.578 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T08:16:57.578 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T08:16:57.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T08:16:57.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:16:57.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T08:16:57.611 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:16:57.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:16:57.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.710 INFO:teuthology.orchestra.run.smithi049.stdout:client25144 2024-03-16T08:16:57.710 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:16:57.710 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T08:16:57.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:57 smithi154 ceph-mon[36471]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:57.727 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:16:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:16:57.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.824 INFO:teuthology.orchestra.run.smithi049.stdout:client25144 2024-03-16T08:16:57.824 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:16:57.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:57 smithi129 ceph-mon[36561]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:16:57.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:16:57.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.880 INFO:teuthology.orchestra.run.smithi049.stdout:client25144 2024-03-16T08:16:57.880 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/053f2308-e36c-11ee-95c9-87774f69a715.client25144/status') 2024-03-16T08:16:57.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:16:57.946 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.25144 (3)172.21.15.49:0/2948813070 2024-03-16T08:16:57.946 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-03-16T08:16:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-03-16T08:16:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-03-16T08:16:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:71 bytes copied, 2.2074e-05 s, 3.2 MB/s 2024-03-16T08:16:57.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:16:57.954 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T08:16:57.964 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T08:16:57.964 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:16:57.964 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 -- 172.21.15.49:0/4092521231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 msgr2=0x7fa280108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 --2- 172.21.15.49:0/4092521231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa280108b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa278009ca0 tx=0x7fa27802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 -- 172.21.15.49:0/4092521231 shutdown_connections 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 --2- 172.21.15.49:0/4092521231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa280108b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 --2- 172.21.15.49:0/4092521231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 0x7fa280101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 --2- 172.21.15.49:0/4092521231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 0x7fa280100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 -- 172.21.15.49:0/4092521231 >> 172.21.15.49:0/4092521231 conn(0x7fa2800fbcc0 msgr2=0x7fa2800fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 -- 172.21.15.49:0/4092521231 shutdown_connections 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.073+0000 7fa2853e6640 1 -- 172.21.15.49:0/4092521231 wait complete. 2024-03-16T08:16:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 Processor -- start 2024-03-16T08:16:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 -- start start 2024-03-16T08:16:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 0x7fa280197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 0x7fa280197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010d4f0 con 0x7fa280102300 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010d370 con 0x7fa280100750 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa2853e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010d670 con 0x7fa280101940 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa27f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa27effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 0x7fa280197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa27f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60252/0 (socket says 172.21.15.49:60252) 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa27f7fe640 1 -- 172.21.15.49:0/2828982685 learned_addr learned my addr 172.21.15.49:0/2828982685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.074+0000 7fa27e7fc640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 0x7fa280197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 -- 172.21.15.49:0/2828982685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 msgr2=0x7fa280197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 0x7fa280197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 -- 172.21.15.49:0/2828982685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 msgr2=0x7fa280197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 0x7fa280197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 -- 172.21.15.49:0/2828982685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28019f640 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa27f7fe640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa278037040 tx=0x7fa278032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278041020 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28019f8d0 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa278034cd0 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278039040 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.075+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28019fdc0 con 0x7fa280102300 2024-03-16T08:16:58.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.076+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2801079a0 con 0x7fa280102300 2024-03-16T08:16:58.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.077+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa278044020 con 0x7fa280102300 2024-03-16T08:16:58.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.078+0000 7fa277fff640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 0x7fa26c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.078+0000 7fa27effd640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 0x7fa26c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.078+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fa2780f08d0 con 0x7fa280102300 2024-03-16T08:16:58.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.079+0000 7fa27effd640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 0x7fa26c0a4d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2801017a0 tx=0x7fa270005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.083+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa27804b020 con 0x7fa280102300 2024-03-16T08:16:58.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.306+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fa2800623e0 con 0x7fa280102300 2024-03-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.435+0000 7fa277fff640 1 -- 172.21.15.49:0/2828982685 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7fa2780bda30 con 0x7fa280102300 2024-03-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 msgr2=0x7fa26c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 0x7fa26c0a4d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2801017a0 tx=0x7fa270005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 msgr2=0x7fa28019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa278037040 tx=0x7fa278032f40 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 shutdown_connections 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa26c0a28a0 0x7fa26c0a4d60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa280102300 0x7fa28019efd0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa280101940 0x7fa280197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 --2- 172.21.15.49:0/2828982685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280100750 0x7fa280197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 >> 172.21.15.49:0/2828982685 conn(0x7fa2800fbcc0 msgr2=0x7fa280104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 shutdown_connections 2024-03-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.437+0000 7fa2853e6640 1 -- 172.21.15.49:0/2828982685 wait complete. 2024-03-16T08:16:58.450 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-16T08:16:58.460 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-16T08:16:58.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:58.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- 172.21.15.49:0/197799828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981018f0 msgr2=0x7fec98101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 --2- 172.21.15.49:0/197799828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981018f0 0x7fec98101d70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009900 tx=0x7fec8c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:58.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- 172.21.15.49:0/197799828 shutdown_connections 2024-03-16T08:16:58.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 --2- 172.21.15.49:0/197799828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec981022b0 0x7fec98108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 --2- 172.21.15.49:0/197799828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981018f0 0x7fec98101d70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 --2- 172.21.15.49:0/197799828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec98100700 0x7fec98100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- 172.21.15.49:0/197799828 >> 172.21.15.49:0/197799828 conn(0x7fec980fbc70 msgr2=0x7fec980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- 172.21.15.49:0/197799828 shutdown_connections 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- 172.21.15.49:0/197799828 wait complete. 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 Processor -- start 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.579+0000 7fec9e859640 1 -- start start 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec98100700 0x7fec98197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec981018f0 0x7fec981980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810c360 con 0x7fec981022b0 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810c1e0 con 0x7fec981018f0 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810c4e0 con 0x7fec98100700 2024-03-16T08:16:58.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec97fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec98100700 0x7fec98197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60278/0 (socket says 172.21.15.49:60278) 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 -- 172.21.15.49:0/450990764 learned_addr learned my addr 172.21.15.49:0/450990764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec977fe640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec981018f0 0x7fec981980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 -- 172.21.15.49:0/450990764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec98100700 msgr2=0x7fec98197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec98100700 0x7fec98197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 -- 172.21.15.49:0/450990764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec981018f0 msgr2=0x7fec981980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec981018f0 0x7fec981980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 -- 172.21.15.49:0/450990764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819f7a0 con 0x7fec981022b0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9cdf6640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec90009fd0 tx=0x7fec9000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90017070 con 0x7fec981022b0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90003e50 con 0x7fec981022b0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.580+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819fa90 con 0x7fec981022b0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.581+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900106c0 con 0x7fec981022b0 2024-03-16T08:16:58.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.581+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9819ff60 con 0x7fec981022b0 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.582+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec981a01f0 con 0x7fec981022b0 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.583+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fec90003970 con 0x7fec981022b0 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.583+0000 7fec957fa640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 0x7fec800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.584+0000 7fec97fff640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 0x7fec800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.584+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fec900ce730 con 0x7fec981022b0 2024-03-16T08:16:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.584+0000 7fec97fff640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 0x7fec800a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec98101750 tx=0x7fec8800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.589+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec9009b1b0 con 0x7fec981022b0 2024-03-16T08:16:58.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:58 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T08:16:58.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.786+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec981a08b0 con 0x7fec981022b0 2024-03-16T08:16:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:58 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T08:16:58.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.787+0000 7fec957fa640 1 -- 172.21.15.49:0/450990764 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fec9009f250 con 0x7fec981022b0 2024-03-16T08:16:58.789 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:16:58.789 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 msgr2=0x7fec800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 0x7fec800a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec98101750 tx=0x7fec8800b390 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 msgr2=0x7fec9819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec90009fd0 tx=0x7fec9000b260 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 shutdown_connections 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fec800a2800 0x7fec800a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec981022b0 0x7fec9819f130 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec981018f0 0x7fec981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 --2- 172.21.15.49:0/450990764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec98100700 0x7fec98197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 >> 172.21.15.49:0/450990764 conn(0x7fec980fbc70 msgr2=0x7fec980fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:58.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 shutdown_connections 2024-03-16T08:16:58.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.790+0000 7fec9e859640 1 -- 172.21.15.49:0/450990764 wait complete. 2024-03-16T08:16:58.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:16:58.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:16:58.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:58 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 -- 172.21.15.49:0/3075932380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830102330 msgr2=0x7fb830108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 --2- 172.21.15.49:0/3075932380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830102330 0x7fb830108bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb828009ca0 tx=0x7fb82802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 -- 172.21.15.49:0/3075932380 shutdown_connections 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 --2- 172.21.15.49:0/3075932380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830102330 0x7fb830108bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 --2- 172.21.15.49:0/3075932380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830101970 0x7fb830101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 --2- 172.21.15.49:0/3075932380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830100780 0x7fb830100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 -- 172.21.15.49:0/3075932380 >> 172.21.15.49:0/3075932380 conn(0x7fb8300fbcf0 msgr2=0x7fb8300fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 -- 172.21.15.49:0/3075932380 shutdown_connections 2024-03-16T08:16:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.913+0000 7fb834ac6640 1 -- 172.21.15.49:0/3075932380 wait complete. 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 Processor -- start 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 -- start start 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 0x7fb830198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 0x7fb83019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb83010c370 con 0x7fb830100780 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb83010c1f0 con 0x7fb830102330 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb834ac6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb83010c4f0 con 0x7fb830101970 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb82e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb827fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 0x7fb830198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb82ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 0x7fb83019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb82e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60290/0 (socket says 172.21.15.49:60290) 2024-03-16T08:16:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.914+0000 7fb82e59c640 1 -- 172.21.15.49:0/991194429 learned_addr learned my addr 172.21.15.49:0/991194429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 -- 172.21.15.49:0/991194429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 msgr2=0x7fb830198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 0x7fb830198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 -- 172.21.15.49:0/991194429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 msgr2=0x7fb83019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 0x7fb83019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 -- 172.21.15.49:0/991194429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83019f720 con 0x7fb830100780 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb827fff640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 0x7fb830198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82ed9d640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 0x7fb83019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb82e59c640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb82000e9c0 tx=0x7fb82000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82000cd80 con 0x7fb830100780 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83019fa10 con 0x7fb830100780 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8200035f0 con 0x7fb830100780 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820010690 con 0x7fb830100780 2024-03-16T08:16:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.915+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83019ffb0 con 0x7fb830100780 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.916+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830068a20 con 0x7fb830100780 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.917+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb8200037a0 con 0x7fb830100780 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.918+0000 7fb8277fe640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 0x7fb81c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.918+0000 7fb827fff640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 0x7fb81c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.918+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fb8200ceec0 con 0x7fb830100780 2024-03-16T08:16:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.918+0000 7fb827fff640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 0x7fb81c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb830199010 tx=0x7fb818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:58.921+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb82009c020 con 0x7fb830100780 2024-03-16T08:16:59.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.105+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8301a0480 con 0x7fb830100780 2024-03-16T08:16:59.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.106+0000 7fb8277fe640 1 -- 172.21.15.49:0/991194429 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb82009b920 con 0x7fb830100780 2024-03-16T08:16:59.106 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:16:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 msgr2=0x7fb81c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 0x7fb81c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb830199010 tx=0x7fb818005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 msgr2=0x7fb830197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb82000e9c0 tx=0x7fb82000ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 shutdown_connections 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb830102330 0x7fb83019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb830101970 0x7fb830198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fb81c0a2800 0x7fb81c0a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 --2- 172.21.15.49:0/991194429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb830100780 0x7fb830197b00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 >> 172.21.15.49:0/991194429 conn(0x7fb8300fbcf0 msgr2=0x7fb8300fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 shutdown_connections 2024-03-16T08:16:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.108+0000 7fb834ac6640 1 -- 172.21.15.49:0/991194429 wait complete. 2024-03-16T08:16:59.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 -- 172.21.15.49:0/1871795498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc0100700 msgr2=0x7efcc0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 --2- 172.21.15.49:0/1871795498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc0100700 0x7efcc0100b20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009900 tx=0x7efcbc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 -- 172.21.15.49:0/1871795498 shutdown_connections 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 --2- 172.21.15.49:0/1871795498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 0x7efcc0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 --2- 172.21.15.49:0/1871795498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc01018f0 0x7efcc0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 --2- 172.21.15.49:0/1871795498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc0100700 0x7efcc0100b20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 -- 172.21.15.49:0/1871795498 >> 172.21.15.49:0/1871795498 conn(0x7efcc00fbc90 msgr2=0x7efcc00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 -- 172.21.15.49:0/1871795498 shutdown_connections 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.231+0000 7efcc7eac640 1 -- 172.21.15.49:0/1871795498 wait complete. 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 Processor -- start 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 -- start start 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 0x7efcc006a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 0x7efcc019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010c380 con 0x7efcc01018f0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010c200 con 0x7efcc0100700 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc7eac640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010c500 con 0x7efcc01022b0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60300/0 (socket says 172.21.15.49:60300) 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 -- 172.21.15.49:0/177568358 learned_addr learned my addr 172.21.15.49:0/177568358 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5c48640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 0x7efcc006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc6449640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 0x7efcc019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 -- 172.21.15.49:0/177568358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 msgr2=0x7efcc019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 0x7efcc019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 -- 172.21.15.49:0/177568358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 msgr2=0x7efcc006a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 0x7efcc006a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5447640 1 -- 172.21.15.49:0/177568358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc019f740 con 0x7efcc01018f0 2024-03-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.232+0000 7efcc5c48640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 0x7efcc006a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcc5447640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efcb000ea60 tx=0x7efcb000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000cde0 con 0x7efcc01018f0 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc019fa30 con 0x7efcc01018f0 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb00035f0 con 0x7efcc01018f0 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0010670 con 0x7efcc01018f0 2024-03-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.233+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc019ff80 con 0x7efcc01018f0 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.234+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0107ac0 con 0x7efcc01018f0 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.235+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efcb00037a0 con 0x7efcc01018f0 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.236+0000 7efcb6ffd640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 0x7efcac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.236+0000 7efcc5c48640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 0x7efcac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.236+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7efcb00ced20 con 0x7efcc01018f0 2024-03-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.236+0000 7efcc5c48640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 0x7efcac0a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000c00 tx=0x7efcbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.240+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efcb009bdf0 con 0x7efcc01018f0 2024-03-16T08:16:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.425+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efcc00623e0 con 0x7efcc01018f0 2024-03-16T08:16:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.425+0000 7efcb6ffd640 1 -- 172.21.15.49:0/177568358 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7efcb009b6f0 con 0x7efcc01018f0 2024-03-16T08:16:59.426 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 msgr2=0x7efcac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 0x7efcac0a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000c00 tx=0x7efcbc005fd0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 msgr2=0x7efcc019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efcb000ea60 tx=0x7efcb000ef20 comp rx=0 tx=0).stop 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 shutdown_connections 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc01022b0 0x7efcc019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7efcac0a2800 0x7efcac0a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc01018f0 0x7efcc019c840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 --2- 172.21.15.49:0/177568358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0100700 0x7efcc006a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.427+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 >> 172.21.15.49:0/177568358 conn(0x7efcc00fbc90 msgr2=0x7efcc0104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.428+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 shutdown_connections 2024-03-16T08:16:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.428+0000 7efcc7eac640 1 -- 172.21.15.49:0/177568358 wait complete. 2024-03-16T08:16:59.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 -- 172.21.15.49:0/2025520241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401022a0 msgr2=0x7ff740108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 --2- 172.21.15.49:0/2025520241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401022a0 0x7ff740108b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff738009ca0 tx=0x7ff73802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 -- 172.21.15.49:0/2025520241 shutdown_connections 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 --2- 172.21.15.49:0/2025520241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401022a0 0x7ff740108b30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 --2- 172.21.15.49:0/2025520241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 --2- 172.21.15.49:0/2025520241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401006f0 0x7ff740100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 -- 172.21.15.49:0/2025520241 >> 172.21.15.49:0/2025520241 conn(0x7ff7400fbc80 msgr2=0x7ff7400fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 -- 172.21.15.49:0/2025520241 shutdown_connections 2024-03-16T08:16:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.550+0000 7ff744c8a640 1 -- 172.21.15.49:0/2025520241 wait complete. 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 Processor -- start 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 -- start start 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401022a0 0x7ff74019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff74010c370 con 0x7ff7401006f0 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff74010c1f0 con 0x7ff7401018e0 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff744c8a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff74010c4f0 con 0x7ff7401022a0 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff73e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff73e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60312/0 (socket says 172.21.15.49:60312) 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff73e59c640 1 -- 172.21.15.49:0/1626317563 learned_addr learned my addr 172.21.15.49:0/1626317563 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.551+0000 7ff73dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73ed9d640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401022a0 0x7ff74019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 -- 172.21.15.49:0/1626317563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401022a0 msgr2=0x7ff74019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401022a0 0x7ff74019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 -- 172.21.15.49:0/1626317563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 msgr2=0x7ff740198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 -- 172.21.15.49:0/1626317563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74019f6f0 con 0x7ff7401006f0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73dd9b640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff73e59c640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9c0 tx=0x7ff73000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000cd80 con 0x7ff7401006f0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7300035f0 con 0x7ff7401006f0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74019f9e0 con 0x7ff7401006f0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730010690 con 0x7ff7401006f0 2024-03-16T08:16:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.552+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74019fee0 con 0x7ff7401006f0 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.553+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7401a0170 con 0x7ff7401006f0 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.554+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff7300037a0 con 0x7ff7401006f0 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.555+0000 7ff72f7fe640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 0x7ff7280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.555+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff7300ced90 con 0x7ff7401006f0 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.555+0000 7ff73dd9b640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 0x7ff7280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.555+0000 7ff73dd9b640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 0x7ff7280a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff740198fe0 tx=0x7ff734005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.559+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff73009be60 con 0x7ff7401006f0 2024-03-16T08:16:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T08:16:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:16:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/450990764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/991194429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:16:59 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/177568358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.742+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7401a0340 con 0x7ff7401006f0 2024-03-16T08:16:59.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.743+0000 7ff72f7fe640 1 -- 172.21.15.49:0/1626317563 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff73009b760 con 0x7ff7401006f0 2024-03-16T08:16:59.743 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.744+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 msgr2=0x7ff7280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.744+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 0x7ff7280a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff740198fe0 tx=0x7ff734005f90 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.744+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 msgr2=0x7ff740197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.744+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9c0 tx=0x7ff73000ee80 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 shutdown_connections 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7401022a0 0x7ff74019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7401018e0 0x7ff740198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff7280a2800 0x7ff7280a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 --2- 172.21.15.49:0/1626317563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7401006f0 0x7ff740197ad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 >> 172.21.15.49:0/1626317563 conn(0x7ff7400fbc80 msgr2=0x7ff7400fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 shutdown_connections 2024-03-16T08:16:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.745+0000 7ff744c8a640 1 -- 172.21.15.49:0/1626317563 wait complete. 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/450990764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/991194429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:16:59 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/177568358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.756 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2828982685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/450990764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/991194429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:16:59 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/177568358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- 172.21.15.49:0/227885608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c1022b0 msgr2=0x7fc83c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- 172.21.15.49:0/227885608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c1022b0 0x7fc83c108b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc834009ca0 tx=0x7fc83402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- 172.21.15.49:0/227885608 shutdown_connections 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- 172.21.15.49:0/227885608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c1022b0 0x7fc83c108b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- 172.21.15.49:0/227885608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1018f0 0x7fc83c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- 172.21.15.49:0/227885608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c100700 0x7fc83c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- 172.21.15.49:0/227885608 >> 172.21.15.49:0/227885608 conn(0x7fc83c0fbc70 msgr2=0x7fc83c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- 172.21.15.49:0/227885608 shutdown_connections 2024-03-16T08:16:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- 172.21.15.49:0/227885608 wait complete. 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 Processor -- start 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 -- start start 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.865+0000 7fc8438de640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c1018f0 0x7fc83c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1022b0 0x7fc83c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c10d460 con 0x7fc83c100700 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c10d2e0 con 0x7fc83c1022b0 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c10d5e0 con 0x7fc83c1018f0 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60334/0 (socket says 172.21.15.49:60334) 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 -- 172.21.15.49:0/3174892295 learned_addr learned my addr 172.21.15.49:0/3174892295 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc840e79640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c1018f0 0x7fc83c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc841e7b640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1022b0 0x7fc83c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 -- 172.21.15.49:0/3174892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c1018f0 msgr2=0x7fc83c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c1018f0 0x7fc83c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 -- 172.21.15.49:0/3174892295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1022b0 msgr2=0x7fc83c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1022b0 0x7fc83c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 -- 172.21.15.49:0/3174892295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c19f6c0 con 0x7fc83c100700 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc84167a640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00e9c0 tx=0x7fc82c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c00cd80 con 0x7fc83c100700 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c19f9b0 con 0x7fc83c100700 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc82c0035f0 con 0x7fc83c100700 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c010690 con 0x7fc83c100700 2024-03-16T08:16:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.866+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c19ff00 con 0x7fc83c100700 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.867+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83c0685d0 con 0x7fc83c100700 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.868+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc82c0037a0 con 0x7fc83c100700 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.869+0000 7fc8327fc640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 0x7fc8280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.869+0000 7fc840e79640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 0x7fc8280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.869+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fc82c0cede0 con 0x7fc83c100700 2024-03-16T08:16:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.869+0000 7fc840e79640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 0x7fc8280a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc83c198fb0 tx=0x7fc838005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:16:59.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:16:59.873+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc82c09beb0 con 0x7fc83c100700 2024-03-16T08:17:00.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.058+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc83c0623e0 con 0x7fc83c100700 2024-03-16T08:17:00.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.059+0000 7fc8327fc640 1 -- 172.21.15.49:0/3174892295 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fc82c09b7b0 con 0x7fc83c100700 2024-03-16T08:17:00.060 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:00.060 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 msgr2=0x7fc8280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 0x7fc8280a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc83c198fb0 tx=0x7fc838005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 msgr2=0x7fc83c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00e9c0 tx=0x7fc82c00ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 shutdown_connections 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c1022b0 0x7fc83c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc83c1018f0 0x7fc83c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fc8280a2800 0x7fc8280a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 --2- 172.21.15.49:0/3174892295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc83c100700 0x7fc83c197aa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.061+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 >> 172.21.15.49:0/3174892295 conn(0x7fc83c0fbc70 msgr2=0x7fc83c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.062+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 shutdown_connections 2024-03-16T08:17:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.062+0000 7fc8438de640 1 -- 172.21.15.49:0/3174892295 wait complete. 2024-03-16T08:17:00.073 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:17:00.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.184+0000 7f2ebee9a640 1 -- 172.21.15.49:0/362682187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb8100700 msgr2=0x7f2eb8100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.184+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/362682187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb8100700 0x7f2eb8100b20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009900 tx=0x7f2eac02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- 172.21.15.49:0/362682187 shutdown_connections 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/362682187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 0x7f2eb8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/362682187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb81018f0 0x7f2eb8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/362682187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb8100700 0x7f2eb8100b20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- 172.21.15.49:0/362682187 >> 172.21.15.49:0/362682187 conn(0x7f2eb80fbc90 msgr2=0x7f2eb80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- 172.21.15.49:0/362682187 shutdown_connections 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- 172.21.15.49:0/362682187 wait complete. 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 Processor -- start 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- start start 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb8100700 0x7f2eb806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 0x7f2eb819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c380 con 0x7f2eb81018f0 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c200 con 0x7f2eb8100700 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c500 con 0x7f2eb81022b0 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60352/0 (socket says 172.21.15.49:60352) 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.185+0000 7f2eb7fff640 1 -- 172.21.15.49:0/4261697799 learned_addr learned my addr 172.21.15.49:0/4261697799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2ebcc36640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb8100700 0x7f2eb806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2ebd437640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 0x7f2eb819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 -- 172.21.15.49:0/4261697799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 msgr2=0x7f2eb819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 0x7f2eb819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 -- 172.21.15.49:0/4261697799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb8100700 msgr2=0x7f2eb806a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb8100700 0x7f2eb806a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 -- 172.21.15.49:0/4261697799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb819f760 con 0x7f2eb81018f0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb7fff640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400e9f0 tx=0x7f2ea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400cd80 con 0x7f2eb81018f0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb819fa50 con 0x7f2eb81018f0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea4010910 con 0x7f2eb81018f0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4018990 con 0x7f2eb81018f0 2024-03-16T08:17:00.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.186+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb819ffa0 con 0x7f2eb81018f0 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.187+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb8107aa0 con 0x7f2eb81018f0 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.188+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2ea4010ac0 con 0x7f2eb81018f0 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.189+0000 7f2eb5ffb640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 0x7f2ea80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.189+0000 7f2ebcc36640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 0x7f2ea80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.189+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f2ea40cf150 con 0x7f2eb81018f0 2024-03-16T08:17:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.190+0000 7f2ebcc36640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 0x7f2ea80a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.192+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ea409bbd0 con 0x7f2eb81018f0 2024-03-16T08:17:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.375+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2eb802d400 con 0x7f2eb81018f0 2024-03-16T08:17:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.375+0000 7f2eb5ffb640 1 -- 172.21.15.49:0/4261697799 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f2ea409fc70 con 0x7f2eb81018f0 2024-03-16T08:17:00.376 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 msgr2=0x7f2ea80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 0x7f2ea80a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 msgr2=0x7f2eb819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400e9f0 tx=0x7f2ea400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 shutdown_connections 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb81022b0 0x7f2eb819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f2ea80a2800 0x7f2ea80a4cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb81018f0 0x7f2eb819c860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 --2- 172.21.15.49:0/4261697799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eb8100700 0x7f2eb806a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 >> 172.21.15.49:0/4261697799 conn(0x7f2eb80fbc90 msgr2=0x7f2eb8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 shutdown_connections 2024-03-16T08:17:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.377+0000 7f2ebee9a640 1 -- 172.21.15.49:0/4261697799 wait complete. 2024-03-16T08:17:00.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 -- 172.21.15.49:0/1523977168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19601033b0 msgr2=0x7f1960109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 --2- 172.21.15.49:0/1523977168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19601033b0 0x7f1960109c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ca0 tx=0x7f195802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 -- 172.21.15.49:0/1523977168 shutdown_connections 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 --2- 172.21.15.49:0/1523977168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19601033b0 0x7f1960109c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 --2- 172.21.15.49:0/1523977168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601029f0 0x7f1960102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 --2- 172.21.15.49:0/1523977168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1960101800 0x7f1960101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 -- 172.21.15.49:0/1523977168 >> 172.21.15.49:0/1523977168 conn(0x7f19600fcd70 msgr2=0x7f19600ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 -- 172.21.15.49:0/1523977168 shutdown_connections 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.501+0000 7f1965777640 1 -- 172.21.15.49:0/1523977168 wait complete. 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 Processor -- start 2024-03-16T08:17:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 -- start start 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 0x7f196019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601033b0 0x7f19601a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d480 con 0x7f1960101800 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d300 con 0x7f19601029f0 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f1965777640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d600 con 0x7f19601033b0 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f195effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f195effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60370/0 (socket says 172.21.15.49:60370) 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f195e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 0x7f196019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f195effd640 1 -- 172.21.15.49:0/3614371189 learned_addr learned my addr 172.21.15.49:0/3614371189 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.502+0000 7f195f7fe640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601033b0 0x7f19601a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 -- 172.21.15.49:0/3614371189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601033b0 msgr2=0x7f19601a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601033b0 0x7f19601a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 -- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 msgr2=0x7f1960199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 -- 172.21.15.49:0/3614371189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601a18d0 con 0x7f19601029f0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195effd640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f195e7fc640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 0x7f196019a1d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9f0 tx=0x7f195400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195400ccc0 con 0x7f19601029f0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1954005020 con 0x7f19601029f0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540049e0 con 0x7f19601029f0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a1bc0 con 0x7f19601029f0 2024-03-16T08:17:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.503+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a20c0 con 0x7f19601029f0 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.504+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19601a22f0 con 0x7f19601029f0 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.505+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1954005560 con 0x7f19601029f0 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.505+0000 7f1947fff640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.506+0000 7f195effd640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.506+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f19540ce9f0 con 0x7f19601029f0 2024-03-16T08:17:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.506+0000 7f195effd640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 0x7f19480a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f196019ab60 tx=0x7f1950005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.510+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f19540d0e20 con 0x7f19601029f0 2024-03-16T08:17:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.692+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19601a24c0 con 0x7f19601029f0 2024-03-16T08:17:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.692+0000 7f1947fff640 1 -- 172.21.15.49:0/3614371189 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f19540051d0 con 0x7f19601029f0 2024-03-16T08:17:00.693 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 msgr2=0x7f19480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 0x7f19480a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f196019ab60 tx=0x7f1950005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 msgr2=0x7f196019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 0x7f196019a1d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9f0 tx=0x7f195400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 shutdown_connections 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19601033b0 0x7f19601a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19601029f0 0x7f196019a1d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 --2- 172.21.15.49:0/3614371189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1960101800 0x7f1960199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 >> 172.21.15.49:0/3614371189 conn(0x7f19600fcd70 msgr2=0x7f1960073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 shutdown_connections 2024-03-16T08:17:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.695+0000 7f1965777640 1 -- 172.21.15.49:0/3614371189 wait complete. 2024-03-16T08:17:00.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1626317563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:00.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3174892295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:00.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4261697799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:00.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:17:00.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1626317563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:00.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3174892295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:00.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4261697799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:00.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 -- 172.21.15.49:0/302117045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc101800 msgr2=0x7ff4fc101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 --2- 172.21.15.49:0/302117045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc101800 0x7ff4fc101c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009900 tx=0x7ff4ec02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 -- 172.21.15.49:0/302117045 shutdown_connections 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 --2- 172.21.15.49:0/302117045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 --2- 172.21.15.49:0/302117045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 0x7ff4fc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 --2- 172.21.15.49:0/302117045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc101800 0x7ff4fc101c20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 -- 172.21.15.49:0/302117045 >> 172.21.15.49:0/302117045 conn(0x7ff4fc0fcd70 msgr2=0x7ff4fc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 -- 172.21.15.49:0/302117045 shutdown_connections 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.819+0000 7ff501bc4640 1 -- 172.21.15.49:0/302117045 wait complete. 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 Processor -- start 2024-03-16T08:17:00.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 -- start start 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc101800 0x7ff4fc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 0x7ff4fc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc10d450 con 0x7ff4fc1033b0 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc10d2d0 con 0x7ff4fc101800 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff501bc4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc10d5d0 con 0x7ff4fc1029f0 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60388/0 (socket says 172.21.15.49:60388) 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc101800 0x7ff4fc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 -- 172.21.15.49:0/3996510861 learned_addr learned my addr 172.21.15.49:0/3996510861 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4faffd640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 0x7ff4fc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 -- 172.21.15.49:0/3996510861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 msgr2=0x7ff4fc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 0x7ff4fc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 -- 172.21.15.49:0/3996510861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc101800 msgr2=0x7ff4fc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc101800 0x7ff4fc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.820+0000 7ff4fbfff640 1 -- 172.21.15.49:0/3996510861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc19f710 con 0x7ff4fc1033b0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff4fbfff640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400eb10 tx=0x7ff4f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004b20 con 0x7ff4fc1033b0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4004cc0 con 0x7ff4fc1033b0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc19fa00 con 0x7ff4fc1033b0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4010760 con 0x7ff4fc1033b0 2024-03-16T08:17:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.821+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc19ff50 con 0x7ff4fc1033b0 2024-03-16T08:17:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.822+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc108bf0 con 0x7ff4fc1033b0 2024-03-16T08:17:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.823+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff4f4005020 con 0x7ff4fc1033b0 2024-03-16T08:17:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.823+0000 7ff4f8ff9640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 0x7ff4e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.824+0000 7ff4fb7fe640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 0x7ff4e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.824+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff4f40cef10 con 0x7ff4fc1033b0 2024-03-16T08:17:00.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.824+0000 7ff4fb7fe640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 0x7ff4e40a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000c00 tx=0x7ff4ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:00.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:00.828+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff4f409bfe0 con 0x7ff4fc1033b0 2024-03-16T08:17:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1626317563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3174892295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4261697799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.010+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4fc0623e0 con 0x7ff4fc1033b0 2024-03-16T08:17:01.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.011+0000 7ff4f8ff9640 1 -- 172.21.15.49:0/3996510861 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff4f409b8e0 con 0x7ff4fc1033b0 2024-03-16T08:17:01.011 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 msgr2=0x7ff4e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 0x7ff4e40a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000c00 tx=0x7ff4ec005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 msgr2=0x7ff4fc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400eb10 tx=0x7ff4f400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 shutdown_connections 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff4e40a2880 0x7ff4e40a4d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4fc1033b0 0x7ff4fc19f0a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4fc1029f0 0x7ff4fc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 --2- 172.21.15.49:0/3996510861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4fc101800 0x7ff4fc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 >> 172.21.15.49:0/3996510861 conn(0x7ff4fc0fcd70 msgr2=0x7ff4fc105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 shutdown_connections 2024-03-16T08:17:01.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.013+0000 7ff501bc4640 1 -- 172.21.15.49:0/3996510861 wait complete. 2024-03-16T08:17:01.024 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-16T08:17:01.024 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:17:01.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- 172.21.15.49:0/3425293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781022b0 msgr2=0x7f4778108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 --2- 172.21.15.49:0/3425293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781022b0 0x7f4778108b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f477002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- 172.21.15.49:0/3425293 shutdown_connections 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 --2- 172.21.15.49:0/3425293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781022b0 0x7f4778108b40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 --2- 172.21.15.49:0/3425293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47781018f0 0x7f4778101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 --2- 172.21.15.49:0/3425293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4778100700 0x7f4778100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- 172.21.15.49:0/3425293 >> 172.21.15.49:0/3425293 conn(0x7f47780fbc70 msgr2=0x7f47780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- 172.21.15.49:0/3425293 shutdown_connections 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- 172.21.15.49:0/3425293 wait complete. 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 Processor -- start 2024-03-16T08:17:01.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.133+0000 7f477e4fe640 1 -- start start 2024-03-16T08:17:01.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 0x7f47780694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 0x7f477819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f477810c360 con 0x7f47781018f0 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f477810c1e0 con 0x7f4778100700 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477e4fe640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f477810c4e0 con 0x7f47781022b0 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60408/0 (socket says 172.21.15.49:60408) 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f4777fff640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 0x7f47780694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 -- 172.21.15.49:0/2630884494 learned_addr learned my addr 172.21.15.49:0/2630884494 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477ca9b640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 0x7f477819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 -- 172.21.15.49:0/2630884494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 msgr2=0x7f477819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 0x7f477819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 -- 172.21.15.49:0/2630884494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 msgr2=0x7f47780694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 0x7f47780694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 -- 172.21.15.49:0/2630884494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477819f720 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f477ca9b640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 0x7f477819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f4777fff640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 0x7f47780694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.134+0000 7f47777fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f476800ea60 tx=0x7f476800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.135+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476800cde0 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.135+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477819f9b0 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.135+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47680035f0 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.135+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768010670 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.135+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477819ff50 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.136+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778107a20 con 0x7f47781018f0 2024-03-16T08:17:01.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.137+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f47680037a0 con 0x7f47781018f0 2024-03-16T08:17:01.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.137+0000 7f47757fa640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 0x7f47600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.138+0000 7f4777fff640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 0x7f47600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.138+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f47680cede0 con 0x7f47781018f0 2024-03-16T08:17:01.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.138+0000 7f4777fff640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 0x7f47600a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4778101750 tx=0x7f476c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.141+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f476809bf40 con 0x7f47781018f0 2024-03-16T08:17:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.366+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477802d370 con 0x7f47781018f0 2024-03-16T08:17:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.366+0000 7f47757fa640 1 -- 172.21.15.49:0/2630884494 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f476809b840 con 0x7f47781018f0 2024-03-16T08:17:01.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:01.367 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:17:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.368+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 msgr2=0x7f47600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.368+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 0x7f47600a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4778101750 tx=0x7f476c005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.368+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 msgr2=0x7f477819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.368+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f476800ea60 tx=0x7f476800ef20 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 shutdown_connections 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781022b0 0x7f477819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f47600a2800 0x7f47600a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47781018f0 0x7f477819c7f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 --2- 172.21.15.49:0/2630884494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778100700 0x7f47780694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 >> 172.21.15.49:0/2630884494 conn(0x7f47780fbc70 msgr2=0x7f4778104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 shutdown_connections 2024-03-16T08:17:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.369+0000 7f477e4fe640 1 -- 172.21.15.49:0/2630884494 wait complete. 2024-03-16T08:17:01.381 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-16T08:17:01.381 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-16T08:17:01.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.490+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3189671528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41029f0 msgr2=0x7f6fd4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.490+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3189671528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41029f0 0x7f6fd4102e70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009900 tx=0x7f6fc402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3189671528 shutdown_connections 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3189671528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41033b0 0x7f6fd4109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3189671528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41029f0 0x7f6fd4102e70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3189671528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 0x7f6fd4101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3189671528 >> 172.21.15.49:0/3189671528 conn(0x7f6fd40fcd90 msgr2=0x7f6fd40ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3189671528 shutdown_connections 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3189671528 wait complete. 2024-03-16T08:17:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 Processor -- start 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- start start 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 0x7f6fd4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41029f0 0x7f6fd4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd410d480 con 0x7f6fd41033b0 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd410d300 con 0x7f6fd4101800 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fdc1c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd410d600 con 0x7f6fd41029f0 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.491+0000 7f6fda763640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60426/0 (socket says 172.21.15.49:60426) 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 -- 172.21.15.49:0/3846023609 learned_addr learned my addr 172.21.15.49:0/3846023609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fd9761640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41029f0 0x7f6fd4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fd9f62640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 0x7f6fd4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 -- 172.21.15.49:0/3846023609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41029f0 msgr2=0x7f6fd4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41029f0 0x7f6fd4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 -- 172.21.15.49:0/3846023609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 msgr2=0x7f6fd4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 0x7f6fd4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 -- 172.21.15.49:0/3846023609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd419f720 con 0x7f6fd41033b0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fda763640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00eb10 tx=0x7f6fcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc004b20 con 0x7f6fd41033b0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fcc004cc0 con 0x7f6fd41033b0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc010760 con 0x7f6fd41033b0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd419fa10 con 0x7f6fd41033b0 2024-03-16T08:17:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.492+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd419fee0 con 0x7f6fd41033b0 2024-03-16T08:17:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.493+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd4108b20 con 0x7f6fd41033b0 2024-03-16T08:17:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.494+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6fcc005020 con 0x7f6fd41033b0 2024-03-16T08:17:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.495+0000 7f6fcaffd640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.495+0000 7f6fd9f62640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.495+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f6fcc0cf110 con 0x7f6fd41033b0 2024-03-16T08:17:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.496+0000 7f6fd9f62640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00095f0 tx=0x7f6fd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.500+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fcc09c1e0 con 0x7f6fd41033b0 2024-03-16T08:17:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.682+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fd4061c20 con 0x7f6fc00a2800 2024-03-16T08:17:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.689+0000 7f6fcaffd640 1 -- 172.21.15.49:0/3846023609 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259048 (secure 0 0 0) 0x7f6fd4061c20 con 0x7f6fc00a2800 2024-03-16T08:17:01.692 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:01.693 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 msgr2=0x7f6fc00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00095f0 tx=0x7f6fd0005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 msgr2=0x7f6fd419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00eb10 tx=0x7f6fcc00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 shutdown_connections 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fd41033b0 0x7f6fd419f0b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fd41029f0 0x7f6fd4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 --2- 172.21.15.49:0/3846023609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fd4101800 0x7f6fd4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 >> 172.21.15.49:0/3846023609 conn(0x7f6fd40fcd90 msgr2=0x7f6fd4105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 shutdown_connections 2024-03-16T08:17:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.692+0000 7f6fdc1c6640 1 -- 172.21.15.49:0/3846023609 wait complete. 2024-03-16T08:17:01.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:01 smithi049 ceph-mon[31942]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:17:01.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3614371189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3996510861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2630884494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:01.704 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-03-16T08:17:01.609234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656039,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":835,"num_read_kb":535,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":780,"ondisk_log_size":780,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356644,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585180,"statfs":{"total":1151940427776,"available":1151575224320,"internally_reserved":0,"allocated":11931648,"data_stored":7471757,"data_compressed":113889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002476"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637038+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:16:33.637038+0000","last_peered":"2024-03-16T08:16:33.637038+0000","last_clean":"2024-03-16T08:16:33.637038+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:16:33.637038+0000","last_undegraded":"2024-03-16T08:16:33.637038+0000","last_fullsized":"2024-03-16T08:16:33.637038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.626061+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:16:33.626061+0000","last_peered":"2024-03-16T08:16:33.626061+0000","last_clean":"2024-03-16T08:16:33.626061+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:16:33.626061+0000","last_undegraded":"2024-03-16T08:16:33.626061+0000","last_fullsized":"2024-03-16T08:16:33.626061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635069+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:16:33.635069+0000","last_peered":"2024-03-16T08:16:33.635069+0000","last_clean":"2024-03-16T08:16:33.635069+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:16:33.635069+0000","last_undegraded":"2024-03-16T08:16:33.635069+0000","last_fullsized":"2024-03-16T08:16:33.635069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634668+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:16:33.634668+0000","last_peered":"2024-03-16T08:16:33.634668+0000","last_clean":"2024-03-16T08:16:33.634668+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:16:33.634668+0000","last_undegraded":"2024-03-16T08:16:33.634668+0000","last_fullsized":"2024-03-16T08:16:33.634668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616273+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:16:33.616273+0000","last_peered":"2024-03-16T08:16:33.616273+0000","last_clean":"2024-03-16T08:16:33.616273+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.616273+0000","last_undegraded":"2024-03-16T08:16:33.616273+0000","last_fullsized":"2024-03-16T08:16:33.616273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631271+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:16:33.631271+0000","last_peered":"2024-03-16T08:16:33.631271+0000","last_clean":"2024-03-16T08:16:33.631271+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:16:33.631271+0000","last_undegraded":"2024-03-16T08:16:33.631271+0000","last_fullsized":"2024-03-16T08:16:33.631271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622976+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:16:33.622976+0000","last_peered":"2024-03-16T08:16:33.622976+0000","last_clean":"2024-03-16T08:16:33.622976+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:16:33.622976+0000","last_undegraded":"2024-03-16T08:16:33.622976+0000","last_fullsized":"2024-03-16T08:16:33.622976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621910+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:16:33.621910+0000","last_peered":"2024-03-16T08:16:33.621910+0000","last_clean":"2024-03-16T08:16:33.621910+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:16:33.621910+0000","last_undegraded":"2024-03-16T08:16:33.621910+0000","last_fullsized":"2024-03-16T08:16:33.621910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631822+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:16:33.631822+0000","last_peered":"2024-03-16T08:16:33.631822+0000","last_clean":"2024-03-16T08:16:33.631822+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:16:33.631822+0000","last_undegraded":"2024-03-16T08:16:33.631822+0000","last_fullsized":"2024-03-16T08:16:33.631822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631465+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:16:33.631465+0000","last_peered":"2024-03-16T08:16:33.631465+0000","last_clean":"2024-03-16T08:16:33.631465+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:16:33.631465+0000","last_undegraded":"2024-03-16T08:16:33.631465+0000","last_fullsized":"2024-03-16T08:16:33.631465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637825+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:16:33.637825+0000","last_peered":"2024-03-16T08:16:33.637825+0000","last_clean":"2024-03-16T08:16:33.637825+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:16:33.637825+0000","last_undegraded":"2024-03-16T08:16:33.637825+0000","last_fullsized":"2024-03-16T08:16:33.637825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634424+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:16:33.634424+0000","last_peered":"2024-03-16T08:16:33.634424+0000","last_clean":"2024-03-16T08:16:33.634424+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:16:33.634424+0000","last_undegraded":"2024-03-16T08:16:33.634424+0000","last_fullsized":"2024-03-16T08:16:33.634424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615731+0000","last_change":"2024-03-16T08:16:05.389606+0000","last_active":"2024-03-16T08:16:33.615731+0000","last_peered":"2024-03-16T08:16:33.615731+0000","last_clean":"2024-03-16T08:16:33.615731+0000","last_became_active":"2024-03-16T08:16:05.389062+0000","last_became_peered":"2024-03-16T08:16:05.389062+0000","last_unstale":"2024-03-16T08:16:33.615731+0000","last_undegraded":"2024-03-16T08:16:33.615731+0000","last_fullsized":"2024-03-16T08:16:33.615731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:04:08.093332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623216+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:16:33.623216+0000","last_peered":"2024-03-16T08:16:33.623216+0000","last_clean":"2024-03-16T08:16:33.623216+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:16:33.623216+0000","last_undegraded":"2024-03-16T08:16:33.623216+0000","last_fullsized":"2024-03-16T08:16:33.623216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632440+0000","last_change":"2024-03-16T08:16:05.654998+0000","last_active":"2024-03-16T08:16:33.632440+0000","last_peered":"2024-03-16T08:16:33.632440+0000","last_clean":"2024-03-16T08:16:33.632440+0000","last_became_active":"2024-03-16T08:16:05.654444+0000","last_became_peered":"2024-03-16T08:16:05.654444+0000","last_unstale":"2024-03-16T08:16:33.632440+0000","last_undegraded":"2024-03-16T08:16:33.632440+0000","last_fullsized":"2024-03-16T08:16:33.632440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:04.343402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637327+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:16:33.637327+0000","last_peered":"2024-03-16T08:16:33.637327+0000","last_clean":"2024-03-16T08:16:33.637327+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:16:33.637327+0000","last_undegraded":"2024-03-16T08:16:33.637327+0000","last_fullsized":"2024-03-16T08:16:33.637327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637250+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:16:33.637250+0000","last_peered":"2024-03-16T08:16:33.637250+0000","last_clean":"2024-03-16T08:16:33.637250+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.637250+0000","last_undegraded":"2024-03-16T08:16:33.637250+0000","last_fullsized":"2024-03-16T08:16:33.637250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639565+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:16:33.639565+0000","last_peered":"2024-03-16T08:16:33.639565+0000","last_clean":"2024-03-16T08:16:33.639565+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:16:33.639565+0000","last_undegraded":"2024-03-16T08:16:33.639565+0000","last_fullsized":"2024-03-16T08:16:33.639565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621658+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:16:33.621658+0000","last_peered":"2024-03-16T08:16:33.621658+0000","last_clean":"2024-03-16T08:16:33.621658+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:16:33.621658+0000","last_undegraded":"2024-03-16T08:16:33.621658+0000","last_fullsized":"2024-03-16T08:16:33.621658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380775+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:16:05.380775+0000","last_peered":"2024-03-16T08:16:05.380775+0000","last_clean":"2024-03-16T08:16:05.380775+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:16:05.380775+0000","last_undegraded":"2024-03-16T08:16:05.380775+0000","last_fullsized":"2024-03-16T08:16:05.380775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622106+0000","last_change":"2024-03-16T08:16:05.654774+0000","last_active":"2024-03-16T08:16:33.622106+0000","last_peered":"2024-03-16T08:16:33.622106+0000","last_clean":"2024-03-16T08:16:33.622106+0000","last_became_active":"2024-03-16T08:16:05.654297+0000","last_became_peered":"2024-03-16T08:16:05.654297+0000","last_unstale":"2024-03-16T08:16:33.622106+0000","last_undegraded":"2024-03-16T08:16:33.622106+0000","last_fullsized":"2024-03-16T08:16:33.622106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:50.079111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386863+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:16:05.386863+0000","last_peered":"2024-03-16T08:16:05.386863+0000","last_clean":"2024-03-16T08:16:05.386863+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:16:05.386863+0000","last_undegraded":"2024-03-16T08:16:05.386863+0000","last_fullsized":"2024-03-16T08:16:05.386863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635060+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:16:33.635060+0000","last_peered":"2024-03-16T08:16:33.635060+0000","last_clean":"2024-03-16T08:16:33.635060+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:16:33.635060+0000","last_undegraded":"2024-03-16T08:16:33.635060+0000","last_fullsized":"2024-03-16T08:16:33.635060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388254+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:16:05.388254+0000","last_peered":"2024-03-16T08:16:05.388254+0000","last_clean":"2024-03-16T08:16:05.388254+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:16:05.388254+0000","last_undegraded":"2024-03-16T08:16:05.388254+0000","last_fullsized":"2024-03-16T08:16:05.388254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637196+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:16:33.637196+0000","last_peered":"2024-03-16T08:16:33.637196+0000","last_clean":"2024-03-16T08:16:33.637196+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:16:33.637196+0000","last_undegraded":"2024-03-16T08:16:33.637196+0000","last_fullsized":"2024-03-16T08:16:33.637196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388257+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:16:05.388257+0000","last_peered":"2024-03-16T08:16:05.388257+0000","last_clean":"2024-03-16T08:16:05.388257+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:16:05.388257+0000","last_undegraded":"2024-03-16T08:16:05.388257+0000","last_fullsized":"2024-03-16T08:16:05.388257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634147+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:16:33.634147+0000","last_peered":"2024-03-16T08:16:33.634147+0000","last_clean":"2024-03-16T08:16:33.634147+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:16:33.634147+0000","last_undegraded":"2024-03-16T08:16:33.634147+0000","last_fullsized":"2024-03-16T08:16:33.634147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.698827+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:16:59.698827+0000","last_peered":"2024-03-16T08:16:59.698827+0000","last_clean":"2024-03-16T08:16:59.698827+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:16:59.698827+0000","last_undegraded":"2024-03-16T08:16:59.698827+0000","last_fullsized":"2024-03-16T08:16:59.698827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616873+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:16:33.616873+0000","last_peered":"2024-03-16T08:16:33.616873+0000","last_clean":"2024-03-16T08:16:33.616873+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:16:33.616873+0000","last_undegraded":"2024-03-16T08:16:33.616873+0000","last_fullsized":"2024-03-16T08:16:33.616873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383442+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:16:05.383442+0000","last_peered":"2024-03-16T08:16:05.383442+0000","last_clean":"2024-03-16T08:16:05.383442+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:16:05.383442+0000","last_undegraded":"2024-03-16T08:16:05.383442+0000","last_fullsized":"2024-03-16T08:16:05.383442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634056+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:16:33.634056+0000","last_peered":"2024-03-16T08:16:33.634056+0000","last_clean":"2024-03-16T08:16:33.634056+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:16:33.634056+0000","last_undegraded":"2024-03-16T08:16:33.634056+0000","last_fullsized":"2024-03-16T08:16:33.634056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385161+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:16:05.385161+0000","last_peered":"2024-03-16T08:16:05.385161+0000","last_clean":"2024-03-16T08:16:05.385161+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:16:05.385161+0000","last_undegraded":"2024-03-16T08:16:05.385161+0000","last_fullsized":"2024-03-16T08:16:05.385161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624699+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:16:33.624699+0000","last_peered":"2024-03-16T08:16:33.624699+0000","last_clean":"2024-03-16T08:16:33.624699+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:16:33.624699+0000","last_undegraded":"2024-03-16T08:16:33.624699+0000","last_fullsized":"2024-03-16T08:16:33.624699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387954+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:16:05.387954+0000","last_peered":"2024-03-16T08:16:05.387954+0000","last_clean":"2024-03-16T08:16:05.387954+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:16:05.387954+0000","last_undegraded":"2024-03-16T08:16:05.387954+0000","last_fullsized":"2024-03-16T08:16:05.387954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625831+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:16:33.625831+0000","last_peered":"2024-03-16T08:16:33.625831+0000","last_clean":"2024-03-16T08:16:33.625831+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:16:33.625831+0000","last_undegraded":"2024-03-16T08:16:33.625831+0000","last_fullsized":"2024-03-16T08:16:33.625831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:55.693854+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:16:55.693854+0000","last_peered":"2024-03-16T08:16:55.693854+0000","last_clean":"2024-03-16T08:16:55.693854+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:16:55.693854+0000","last_undegraded":"2024-03-16T08:16:55.693854+0000","last_fullsized":"2024-03-16T08:16:55.693854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638002+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:16:33.638002+0000","last_peered":"2024-03-16T08:16:33.638002+0000","last_clean":"2024-03-16T08:16:33.638002+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:16:33.638002+0000","last_undegraded":"2024-03-16T08:16:33.638002+0000","last_fullsized":"2024-03-16T08:16:33.638002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381137+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:16:05.381137+0000","last_peered":"2024-03-16T08:16:05.381137+0000","last_clean":"2024-03-16T08:16:05.381137+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:16:05.381137+0000","last_undegraded":"2024-03-16T08:16:05.381137+0000","last_fullsized":"2024-03-16T08:16:05.381137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630292+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:16:33.630292+0000","last_peered":"2024-03-16T08:16:33.630292+0000","last_clean":"2024-03-16T08:16:33.630292+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:16:33.630292+0000","last_undegraded":"2024-03-16T08:16:33.630292+0000","last_fullsized":"2024-03-16T08:16:33.630292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389097+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:16:05.389097+0000","last_peered":"2024-03-16T08:16:05.389097+0000","last_clean":"2024-03-16T08:16:05.389097+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:16:05.389097+0000","last_undegraded":"2024-03-16T08:16:05.389097+0000","last_fullsized":"2024-03-16T08:16:05.389097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639262+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:16:33.639262+0000","last_peered":"2024-03-16T08:16:33.639262+0000","last_clean":"2024-03-16T08:16:33.639262+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:16:33.639262+0000","last_undegraded":"2024-03-16T08:16:33.639262+0000","last_fullsized":"2024-03-16T08:16:33.639262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":279,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:56.694997+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:16:56.694997+0000","last_peered":"2024-03-16T08:16:56.694997+0000","last_clean":"2024-03-16T08:16:56.694997+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:16:56.694997+0000","last_undegraded":"2024-03-16T08:16:56.694997+0000","last_fullsized":"2024-03-16T08:16:56.694997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":174,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629485+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:16:33.629485+0000","last_peered":"2024-03-16T08:16:33.629485+0000","last_clean":"2024-03-16T08:16:33.629485+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:16:33.629485+0000","last_undegraded":"2024-03-16T08:16:33.629485+0000","last_fullsized":"2024-03-16T08:16:33.629485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653483+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:16:05.653483+0000","last_peered":"2024-03-16T08:16:05.653483+0000","last_clean":"2024-03-16T08:16:05.653483+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:16:05.653483+0000","last_undegraded":"2024-03-16T08:16:05.653483+0000","last_fullsized":"2024-03-16T08:16:05.653483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629218+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:16:33.629218+0000","last_peered":"2024-03-16T08:16:33.629218+0000","last_clean":"2024-03-16T08:16:33.629218+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:16:33.629218+0000","last_undegraded":"2024-03-16T08:16:33.629218+0000","last_fullsized":"2024-03-16T08:16:33.629218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388595+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:16:05.388595+0000","last_peered":"2024-03-16T08:16:05.388595+0000","last_clean":"2024-03-16T08:16:05.388595+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:16:05.388595+0000","last_undegraded":"2024-03-16T08:16:05.388595+0000","last_fullsized":"2024-03-16T08:16:05.388595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628965+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:16:33.628965+0000","last_peered":"2024-03-16T08:16:33.628965+0000","last_clean":"2024-03-16T08:16:33.628965+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:16:33.628965+0000","last_undegraded":"2024-03-16T08:16:33.628965+0000","last_fullsized":"2024-03-16T08:16:33.628965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.716883+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:16:59.716883+0000","last_peered":"2024-03-16T08:16:59.716883+0000","last_clean":"2024-03-16T08:16:59.716883+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:16:59.716883+0000","last_undegraded":"2024-03-16T08:16:59.716883+0000","last_fullsized":"2024-03-16T08:16:59.716883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615961+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:16:33.615961+0000","last_peered":"2024-03-16T08:16:33.615961+0000","last_clean":"2024-03-16T08:16:33.615961+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:16:33.615961+0000","last_undegraded":"2024-03-16T08:16:33.615961+0000","last_fullsized":"2024-03-16T08:16:33.615961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387994+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:16:05.387994+0000","last_peered":"2024-03-16T08:16:05.387994+0000","last_clean":"2024-03-16T08:16:05.387994+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:16:05.387994+0000","last_undegraded":"2024-03-16T08:16:05.387994+0000","last_fullsized":"2024-03-16T08:16:05.387994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636796+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:16:33.636796+0000","last_peered":"2024-03-16T08:16:33.636796+0000","last_clean":"2024-03-16T08:16:33.636796+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:16:33.636796+0000","last_undegraded":"2024-03-16T08:16:33.636796+0000","last_fullsized":"2024-03-16T08:16:33.636796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.394865+0000","last_change":"2024-03-16T08:16:05.394457+0000","last_active":"2024-03-16T08:16:05.394865+0000","last_peered":"2024-03-16T08:16:05.394865+0000","last_clean":"2024-03-16T08:16:05.394865+0000","last_became_active":"2024-03-16T08:16:05.390239+0000","last_became_peered":"2024-03-16T08:16:05.390239+0000","last_unstale":"2024-03-16T08:16:05.394865+0000","last_undegraded":"2024-03-16T08:16:05.394865+0000","last_fullsized":"2024-03-16T08:16:05.394865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:29:29.442138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631505+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:16:33.631505+0000","last_peered":"2024-03-16T08:16:33.631505+0000","last_clean":"2024-03-16T08:16:33.631505+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:16:33.631505+0000","last_undegraded":"2024-03-16T08:16:33.631505+0000","last_fullsized":"2024-03-16T08:16:33.631505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.390744+0000","last_change":"2024-03-16T08:16:05.390744+0000","last_active":"2024-03-16T08:16:05.390744+0000","last_peered":"2024-03-16T08:16:05.390744+0000","last_clean":"2024-03-16T08:16:05.390744+0000","last_became_active":"2024-03-16T08:16:05.390378+0000","last_became_peered":"2024-03-16T08:16:05.390378+0000","last_unstale":"2024-03-16T08:16:05.390744+0000","last_undegraded":"2024-03-16T08:16:05.390744+0000","last_fullsized":"2024-03-16T08:16:05.390744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:13:29.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.617438+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:16:33.617438+0000","last_peered":"2024-03-16T08:16:33.617438+0000","last_clean":"2024-03-16T08:16:33.617438+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:16:33.617438+0000","last_undegraded":"2024-03-16T08:16:33.617438+0000","last_fullsized":"2024-03-16T08:16:33.617438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380895+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:16:05.380895+0000","last_peered":"2024-03-16T08:16:05.380895+0000","last_clean":"2024-03-16T08:16:05.380895+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:16:05.380895+0000","last_undegraded":"2024-03-16T08:16:05.380895+0000","last_fullsized":"2024-03-16T08:16:05.380895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.620223+0000","last_change":"2024-03-16T08:16:05.389020+0000","last_active":"2024-03-16T08:16:33.620223+0000","last_peered":"2024-03-16T08:16:33.620223+0000","last_clean":"2024-03-16T08:16:33.620223+0000","last_became_active":"2024-03-16T08:16:05.388668+0000","last_became_peered":"2024-03-16T08:16:05.388668+0000","last_unstale":"2024-03-16T08:16:33.620223+0000","last_undegraded":"2024-03-16T08:16:33.620223+0000","last_fullsized":"2024-03-16T08:16:33.620223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:34:04.661388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633962+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:16:33.633962+0000","last_peered":"2024-03-16T08:16:33.633962+0000","last_clean":"2024-03-16T08:16:33.633962+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:16:33.633962+0000","last_undegraded":"2024-03-16T08:16:33.633962+0000","last_fullsized":"2024-03-16T08:16:33.633962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":136,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:58.715510+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:16:58.715510+0000","last_peered":"2024-03-16T08:16:58.715510+0000","last_clean":"2024-03-16T08:16:58.715510+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:16:58.715510+0000","last_undegraded":"2024-03-16T08:16:58.715510+0000","last_fullsized":"2024-03-16T08:16:58.715510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636330+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:16:33.636330+0000","last_peered":"2024-03-16T08:16:33.636330+0000","last_clean":"2024-03-16T08:16:33.636330+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:16:33.636330+0000","last_undegraded":"2024-03-16T08:16:33.636330+0000","last_fullsized":"2024-03-16T08:16:33.636330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387927+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:16:05.387927+0000","last_peered":"2024-03-16T08:16:05.387927+0000","last_clean":"2024-03-16T08:16:05.387927+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:16:05.387927+0000","last_undegraded":"2024-03-16T08:16:05.387927+0000","last_fullsized":"2024-03-16T08:16:05.387927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380865+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:16:05.380865+0000","last_peered":"2024-03-16T08:16:05.380865+0000","last_clean":"2024-03-16T08:16:05.380865+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:16:05.380865+0000","last_undegraded":"2024-03-16T08:16:05.380865+0000","last_fullsized":"2024-03-16T08:16:05.380865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.627112+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:16:33.627112+0000","last_peered":"2024-03-16T08:16:33.627112+0000","last_clean":"2024-03-16T08:16:33.627112+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:16:33.627112+0000","last_undegraded":"2024-03-16T08:16:33.627112+0000","last_fullsized":"2024-03-16T08:16:33.627112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380918+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:16:05.380918+0000","last_peered":"2024-03-16T08:16:05.380918+0000","last_clean":"2024-03-16T08:16:05.380918+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:16:05.380918+0000","last_undegraded":"2024-03-16T08:16:05.380918+0000","last_fullsized":"2024-03-16T08:16:05.380918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388079+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:16:05.388079+0000","last_peered":"2024-03-16T08:16:05.388079+0000","last_clean":"2024-03-16T08:16:05.388079+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:16:05.388079+0000","last_undegraded":"2024-03-16T08:16:05.388079+0000","last_fullsized":"2024-03-16T08:16:05.388079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616704+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:16:33.616704+0000","last_peered":"2024-03-16T08:16:33.616704+0000","last_clean":"2024-03-16T08:16:33.616704+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:16:33.616704+0000","last_undegraded":"2024-03-16T08:16:33.616704+0000","last_fullsized":"2024-03-16T08:16:33.616704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380845+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:16:05.380845+0000","last_peered":"2024-03-16T08:16:05.380845+0000","last_clean":"2024-03-16T08:16:05.380845+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:16:05.380845+0000","last_undegraded":"2024-03-16T08:16:05.380845+0000","last_fullsized":"2024-03-16T08:16:05.380845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624878+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:16:33.624878+0000","last_peered":"2024-03-16T08:16:33.624878+0000","last_clean":"2024-03-16T08:16:33.624878+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:16:33.624878+0000","last_undegraded":"2024-03-16T08:16:33.624878+0000","last_fullsized":"2024-03-16T08:16:33.624878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388054+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:16:05.388054+0000","last_peered":"2024-03-16T08:16:05.388054+0000","last_clean":"2024-03-16T08:16:05.388054+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:16:05.388054+0000","last_undegraded":"2024-03-16T08:16:05.388054+0000","last_fullsized":"2024-03-16T08:16:05.388054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632266+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:16:33.632266+0000","last_peered":"2024-03-16T08:16:33.632266+0000","last_clean":"2024-03-16T08:16:33.632266+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:16:33.632266+0000","last_undegraded":"2024-03-16T08:16:33.632266+0000","last_fullsized":"2024-03-16T08:16:33.632266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652991+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:16:05.652991+0000","last_peered":"2024-03-16T08:16:05.652991+0000","last_clean":"2024-03-16T08:16:05.652991+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:16:05.652991+0000","last_undegraded":"2024-03-16T08:16:05.652991+0000","last_fullsized":"2024-03-16T08:16:05.652991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.585204+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:16:33.585204+0000","last_peered":"2024-03-16T08:16:33.585204+0000","last_clean":"2024-03-16T08:16:33.585204+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:16:33.585204+0000","last_undegraded":"2024-03-16T08:16:33.585204+0000","last_fullsized":"2024-03-16T08:16:33.585204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384150+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:16:05.384150+0000","last_peered":"2024-03-16T08:16:05.384150+0000","last_clean":"2024-03-16T08:16:05.384150+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:16:05.384150+0000","last_undegraded":"2024-03-16T08:16:05.384150+0000","last_fullsized":"2024-03-16T08:16:05.384150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625869+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:16:33.625869+0000","last_peered":"2024-03-16T08:16:33.625869+0000","last_clean":"2024-03-16T08:16:33.625869+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:16:33.625869+0000","last_undegraded":"2024-03-16T08:16:33.625869+0000","last_fullsized":"2024-03-16T08:16:33.625869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653334+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:16:05.653334+0000","last_peered":"2024-03-16T08:16:05.653334+0000","last_clean":"2024-03-16T08:16:05.653334+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:16:05.653334+0000","last_undegraded":"2024-03-16T08:16:05.653334+0000","last_fullsized":"2024-03-16T08:16:05.653334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628250+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:16:33.628250+0000","last_peered":"2024-03-16T08:16:33.628250+0000","last_clean":"2024-03-16T08:16:33.628250+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:16:33.628250+0000","last_undegraded":"2024-03-16T08:16:33.628250+0000","last_fullsized":"2024-03-16T08:16:33.628250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387802+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:16:05.387802+0000","last_peered":"2024-03-16T08:16:05.387802+0000","last_clean":"2024-03-16T08:16:05.387802+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:16:05.387802+0000","last_undegraded":"2024-03-16T08:16:05.387802+0000","last_fullsized":"2024-03-16T08:16:05.387802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637947+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:16:33.637947+0000","last_peered":"2024-03-16T08:16:33.637947+0000","last_clean":"2024-03-16T08:16:33.637947+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:16:33.637947+0000","last_undegraded":"2024-03-16T08:16:33.637947+0000","last_fullsized":"2024-03-16T08:16:33.637947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653465+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:16:05.653465+0000","last_peered":"2024-03-16T08:16:05.653465+0000","last_clean":"2024-03-16T08:16:05.653465+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:16:05.653465+0000","last_undegraded":"2024-03-16T08:16:05.653465+0000","last_fullsized":"2024-03-16T08:16:05.653465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638034+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:16:33.638034+0000","last_peered":"2024-03-16T08:16:33.638034+0000","last_clean":"2024-03-16T08:16:33.638034+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:16:33.638034+0000","last_undegraded":"2024-03-16T08:16:33.638034+0000","last_fullsized":"2024-03-16T08:16:33.638034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388862+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:16:05.388862+0000","last_peered":"2024-03-16T08:16:05.388862+0000","last_clean":"2024-03-16T08:16:05.388862+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:16:05.388862+0000","last_undegraded":"2024-03-16T08:16:05.388862+0000","last_fullsized":"2024-03-16T08:16:05.388862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.619455+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:16:33.619455+0000","last_peered":"2024-03-16T08:16:33.619455+0000","last_clean":"2024-03-16T08:16:33.619455+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:16:33.619455+0000","last_undegraded":"2024-03-16T08:16:33.619455+0000","last_fullsized":"2024-03-16T08:16:33.619455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384729+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:16:05.384729+0000","last_peered":"2024-03-16T08:16:05.384729+0000","last_clean":"2024-03-16T08:16:05.384729+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:16:05.384729+0000","last_undegraded":"2024-03-16T08:16:05.384729+0000","last_fullsized":"2024-03-16T08:16:05.384729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630956+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:16:33.630956+0000","last_peered":"2024-03-16T08:16:33.630956+0000","last_clean":"2024-03-16T08:16:33.630956+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:16:33.630956+0000","last_undegraded":"2024-03-16T08:16:33.630956+0000","last_fullsized":"2024-03-16T08:16:33.630956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624759+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:16:33.624759+0000","last_peered":"2024-03-16T08:16:33.624759+0000","last_clean":"2024-03-16T08:16:33.624759+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:16:33.624759+0000","last_undegraded":"2024-03-16T08:16:33.624759+0000","last_fullsized":"2024-03-16T08:16:33.624759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631321+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:16:33.631321+0000","last_peered":"2024-03-16T08:16:33.631321+0000","last_clean":"2024-03-16T08:16:33.631321+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:16:33.631321+0000","last_undegraded":"2024-03-16T08:16:33.631321+0000","last_fullsized":"2024-03-16T08:16:33.631321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622948+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:16:33.622948+0000","last_peered":"2024-03-16T08:16:33.622948+0000","last_clean":"2024-03-16T08:16:33.622948+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:16:33.622948+0000","last_undegraded":"2024-03-16T08:16:33.622948+0000","last_fullsized":"2024-03-16T08:16:33.622948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633266+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:16:33.633266+0000","last_peered":"2024-03-16T08:16:33.633266+0000","last_clean":"2024-03-16T08:16:33.633266+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:16:33.633266+0000","last_undegraded":"2024-03-16T08:16:33.633266+0000","last_fullsized":"2024-03-16T08:16:33.633266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632885+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:16:33.632885+0000","last_peered":"2024-03-16T08:16:33.632885+0000","last_clean":"2024-03-16T08:16:33.632885+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:16:33.632885+0000","last_undegraded":"2024-03-16T08:16:33.632885+0000","last_fullsized":"2024-03-16T08:16:33.632885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623433+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:16:33.623433+0000","last_peered":"2024-03-16T08:16:33.623433+0000","last_clean":"2024-03-16T08:16:33.623433+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:16:33.623433+0000","last_undegraded":"2024-03-16T08:16:33.623433+0000","last_fullsized":"2024-03-16T08:16:33.623433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639557+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:16:33.639557+0000","last_peered":"2024-03-16T08:16:33.639557+0000","last_clean":"2024-03-16T08:16:33.639557+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:16:33.639557+0000","last_undegraded":"2024-03-16T08:16:33.639557+0000","last_fullsized":"2024-03-16T08:16:33.639557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616343+0000","last_change":"2024-03-16T08:16:05.654782+0000","last_active":"2024-03-16T08:16:33.616343+0000","last_peered":"2024-03-16T08:16:33.616343+0000","last_clean":"2024-03-16T08:16:33.616343+0000","last_became_active":"2024-03-16T08:16:05.654117+0000","last_became_peered":"2024-03-16T08:16:05.654117+0000","last_unstale":"2024-03-16T08:16:33.616343+0000","last_undegraded":"2024-03-16T08:16:33.616343+0000","last_fullsized":"2024-03-16T08:16:33.616343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:17:45.678541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.614995+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:16:33.614995+0000","last_peered":"2024-03-16T08:16:33.614995+0000","last_clean":"2024-03-16T08:16:33.614995+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:16:33.614995+0000","last_undegraded":"2024-03-16T08:16:33.614995+0000","last_fullsized":"2024-03-16T08:16:33.614995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624368+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:16:33.624368+0000","last_peered":"2024-03-16T08:16:33.624368+0000","last_clean":"2024-03-16T08:16:33.624368+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:16:33.624368+0000","last_undegraded":"2024-03-16T08:16:33.624368+0000","last_fullsized":"2024-03-16T08:16:33.624368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639654+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:16:33.639654+0000","last_peered":"2024-03-16T08:16:33.639654+0000","last_clean":"2024-03-16T08:16:33.639654+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:16:33.639654+0000","last_undegraded":"2024-03-16T08:16:33.639654+0000","last_fullsized":"2024-03-16T08:16:33.639654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629430+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:16:33.629430+0000","last_peered":"2024-03-16T08:16:33.629430+0000","last_clean":"2024-03-16T08:16:33.629430+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:16:33.629430+0000","last_undegraded":"2024-03-16T08:16:33.629430+0000","last_fullsized":"2024-03-16T08:16:33.629430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633659+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:16:33.633659+0000","last_peered":"2024-03-16T08:16:33.633659+0000","last_clean":"2024-03-16T08:16:33.633659+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:16:33.633659+0000","last_undegraded":"2024-03-16T08:16:33.633659+0000","last_fullsized":"2024-03-16T08:16:33.633659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65437,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":416,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196619,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":459215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1111969,"data_compressed":37963,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051420,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":520214,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521788,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1051104,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61603,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63177,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62873,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:17:01.708 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T08:17:01.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:01 smithi154 ceph-mon[36471]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:17:01.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3614371189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3996510861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2630884494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:01.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 -- 172.21.15.49:0/353761946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101033c0 msgr2=0x7fdb10109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 --2- 172.21.15.49:0/353761946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101033c0 0x7fdb10109c50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009ca0 tx=0x7fdb0802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 -- 172.21.15.49:0/353761946 shutdown_connections 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 --2- 172.21.15.49:0/353761946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb101033c0 0x7fdb10109c50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 --2- 172.21.15.49:0/353761946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10102a00 0x7fdb10102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 --2- 172.21.15.49:0/353761946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb10101810 0x7fdb10101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 -- 172.21.15.49:0/353761946 >> 172.21.15.49:0/353761946 conn(0x7fdb100fcdc0 msgr2=0x7fdb100ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 -- 172.21.15.49:0/353761946 shutdown_connections 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.852+0000 7fdb180e8640 1 -- 172.21.15.49:0/353761946 wait complete. 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 Processor -- start 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 -- start start 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 0x7fdb10197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb101033c0 0x7fdb1019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d490 con 0x7fdb10102a00 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d310 con 0x7fdb10101810 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb180e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d610 con 0x7fdb101033c0 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15e84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 0x7fdb10197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60458/0 (socket says 172.21.15.49:60458) 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 -- 172.21.15.49:0/927856010 learned_addr learned my addr 172.21.15.49:0/927856010 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb16685640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb101033c0 0x7fdb1019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 -- 172.21.15.49:0/927856010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb101033c0 msgr2=0x7fdb1019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb101033c0 0x7fdb1019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 -- 172.21.15.49:0/927856010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 msgr2=0x7fdb10197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 0x7fdb10197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 -- 172.21.15.49:0/927856010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1019f6d0 con 0x7fdb10102a00 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15e84640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 0x7fdb10197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.853+0000 7fdb15683640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e9f0 tx=0x7fdb0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.854+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00cd80 con 0x7fdb10102a00 2024-03-16T08:17:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.854+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c010910 con 0x7fdb10102a00 2024-03-16T08:17:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.854+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c018990 con 0x7fdb10102a00 2024-03-16T08:17:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.854+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1019f9c0 con 0x7fdb10102a00 2024-03-16T08:17:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.854+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1019ff10 con 0x7fdb10102a00 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.855+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb10108b30 con 0x7fdb10102a00 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.856+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdb0c010ac0 con 0x7fdb10102a00 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.856+0000 7fdb06ffd640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 0x7fdafc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.857+0000 7fdb15e84640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 0x7fdafc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.857+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fdb0c0ced80 con 0x7fdb10102a00 2024-03-16T08:17:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.857+0000 7fdb15e84640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 0x7fdafc0a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb10102860 tx=0x7fdb00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:01 smithi129 ceph-mon[36561]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:17:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3614371189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3996510861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2630884494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:01.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:01.861+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdb0c09b800 con 0x7fdb10102a00 2024-03-16T08:17:02.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.041+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb10061c20 con 0x7fdafc0a2800 2024-03-16T08:17:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.048+0000 7fdb06ffd640 1 -- 172.21.15.49:0/927856010 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259048 (secure 0 0 0) 0x7fdb10061c20 con 0x7fdafc0a2800 2024-03-16T08:17:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:17:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 msgr2=0x7fdafc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 0x7fdafc0a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb10102860 tx=0x7fdb00005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 msgr2=0x7fdb10197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e9f0 tx=0x7fdb0c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 shutdown_connections 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb101033c0 0x7fdb1019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fdafc0a2800 0x7fdafc0a4cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb10102a00 0x7fdb10197ff0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 --2- 172.21.15.49:0/927856010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb10101810 0x7fdb10197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 >> 172.21.15.49:0/927856010 conn(0x7fdb100fcdc0 msgr2=0x7fdb10105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 shutdown_connections 2024-03-16T08:17:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.051+0000 7fdb180e8640 1 -- 172.21.15.49:0/927856010 wait complete. 2024-03-16T08:17:02.062 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-03-16T08:17:01.609234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656039,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":835,"num_read_kb":535,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":780,"ondisk_log_size":780,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356644,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585180,"statfs":{"total":1151940427776,"available":1151575224320,"internally_reserved":0,"allocated":11931648,"data_stored":7471757,"data_compressed":113889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002476"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637038+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:16:33.637038+0000","last_peered":"2024-03-16T08:16:33.637038+0000","last_clean":"2024-03-16T08:16:33.637038+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:16:33.637038+0000","last_undegraded":"2024-03-16T08:16:33.637038+0000","last_fullsized":"2024-03-16T08:16:33.637038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.626061+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:16:33.626061+0000","last_peered":"2024-03-16T08:16:33.626061+0000","last_clean":"2024-03-16T08:16:33.626061+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:16:33.626061+0000","last_undegraded":"2024-03-16T08:16:33.626061+0000","last_fullsized":"2024-03-16T08:16:33.626061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635069+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:16:33.635069+0000","last_peered":"2024-03-16T08:16:33.635069+0000","last_clean":"2024-03-16T08:16:33.635069+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:16:33.635069+0000","last_undegraded":"2024-03-16T08:16:33.635069+0000","last_fullsized":"2024-03-16T08:16:33.635069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634668+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:16:33.634668+0000","last_peered":"2024-03-16T08:16:33.634668+0000","last_clean":"2024-03-16T08:16:33.634668+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:16:33.634668+0000","last_undegraded":"2024-03-16T08:16:33.634668+0000","last_fullsized":"2024-03-16T08:16:33.634668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616273+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:16:33.616273+0000","last_peered":"2024-03-16T08:16:33.616273+0000","last_clean":"2024-03-16T08:16:33.616273+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.616273+0000","last_undegraded":"2024-03-16T08:16:33.616273+0000","last_fullsized":"2024-03-16T08:16:33.616273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631271+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:16:33.631271+0000","last_peered":"2024-03-16T08:16:33.631271+0000","last_clean":"2024-03-16T08:16:33.631271+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:16:33.631271+0000","last_undegraded":"2024-03-16T08:16:33.631271+0000","last_fullsized":"2024-03-16T08:16:33.631271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622976+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:16:33.622976+0000","last_peered":"2024-03-16T08:16:33.622976+0000","last_clean":"2024-03-16T08:16:33.622976+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:16:33.622976+0000","last_undegraded":"2024-03-16T08:16:33.622976+0000","last_fullsized":"2024-03-16T08:16:33.622976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621910+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:16:33.621910+0000","last_peered":"2024-03-16T08:16:33.621910+0000","last_clean":"2024-03-16T08:16:33.621910+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:16:33.621910+0000","last_undegraded":"2024-03-16T08:16:33.621910+0000","last_fullsized":"2024-03-16T08:16:33.621910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631822+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:16:33.631822+0000","last_peered":"2024-03-16T08:16:33.631822+0000","last_clean":"2024-03-16T08:16:33.631822+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:16:33.631822+0000","last_undegraded":"2024-03-16T08:16:33.631822+0000","last_fullsized":"2024-03-16T08:16:33.631822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631465+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:16:33.631465+0000","last_peered":"2024-03-16T08:16:33.631465+0000","last_clean":"2024-03-16T08:16:33.631465+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:16:33.631465+0000","last_undegraded":"2024-03-16T08:16:33.631465+0000","last_fullsized":"2024-03-16T08:16:33.631465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637825+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:16:33.637825+0000","last_peered":"2024-03-16T08:16:33.637825+0000","last_clean":"2024-03-16T08:16:33.637825+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:16:33.637825+0000","last_undegraded":"2024-03-16T08:16:33.637825+0000","last_fullsized":"2024-03-16T08:16:33.637825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634424+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:16:33.634424+0000","last_peered":"2024-03-16T08:16:33.634424+0000","last_clean":"2024-03-16T08:16:33.634424+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:16:33.634424+0000","last_undegraded":"2024-03-16T08:16:33.634424+0000","last_fullsized":"2024-03-16T08:16:33.634424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615731+0000","last_change":"2024-03-16T08:16:05.389606+0000","last_active":"2024-03-16T08:16:33.615731+0000","last_peered":"2024-03-16T08:16:33.615731+0000","last_clean":"2024-03-16T08:16:33.615731+0000","last_became_active":"2024-03-16T08:16:05.389062+0000","last_became_peered":"2024-03-16T08:16:05.389062+0000","last_unstale":"2024-03-16T08:16:33.615731+0000","last_undegraded":"2024-03-16T08:16:33.615731+0000","last_fullsized":"2024-03-16T08:16:33.615731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:04:08.093332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623216+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:16:33.623216+0000","last_peered":"2024-03-16T08:16:33.623216+0000","last_clean":"2024-03-16T08:16:33.623216+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:16:33.623216+0000","last_undegraded":"2024-03-16T08:16:33.623216+0000","last_fullsized":"2024-03-16T08:16:33.623216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632440+0000","last_change":"2024-03-16T08:16:05.654998+0000","last_active":"2024-03-16T08:16:33.632440+0000","last_peered":"2024-03-16T08:16:33.632440+0000","last_clean":"2024-03-16T08:16:33.632440+0000","last_became_active":"2024-03-16T08:16:05.654444+0000","last_became_peered":"2024-03-16T08:16:05.654444+0000","last_unstale":"2024-03-16T08:16:33.632440+0000","last_undegraded":"2024-03-16T08:16:33.632440+0000","last_fullsized":"2024-03-16T08:16:33.632440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:04.343402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637327+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:16:33.637327+0000","last_peered":"2024-03-16T08:16:33.637327+0000","last_clean":"2024-03-16T08:16:33.637327+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:16:33.637327+0000","last_undegraded":"2024-03-16T08:16:33.637327+0000","last_fullsized":"2024-03-16T08:16:33.637327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637250+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:16:33.637250+0000","last_peered":"2024-03-16T08:16:33.637250+0000","last_clean":"2024-03-16T08:16:33.637250+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.637250+0000","last_undegraded":"2024-03-16T08:16:33.637250+0000","last_fullsized":"2024-03-16T08:16:33.637250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639565+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:16:33.639565+0000","last_peered":"2024-03-16T08:16:33.639565+0000","last_clean":"2024-03-16T08:16:33.639565+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:16:33.639565+0000","last_undegraded":"2024-03-16T08:16:33.639565+0000","last_fullsized":"2024-03-16T08:16:33.639565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621658+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:16:33.621658+0000","last_peered":"2024-03-16T08:16:33.621658+0000","last_clean":"2024-03-16T08:16:33.621658+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:16:33.621658+0000","last_undegraded":"2024-03-16T08:16:33.621658+0000","last_fullsized":"2024-03-16T08:16:33.621658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380775+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:16:05.380775+0000","last_peered":"2024-03-16T08:16:05.380775+0000","last_clean":"2024-03-16T08:16:05.380775+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:16:05.380775+0000","last_undegraded":"2024-03-16T08:16:05.380775+0000","last_fullsized":"2024-03-16T08:16:05.380775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622106+0000","last_change":"2024-03-16T08:16:05.654774+0000","last_active":"2024-03-16T08:16:33.622106+0000","last_peered":"2024-03-16T08:16:33.622106+0000","last_clean":"2024-03-16T08:16:33.622106+0000","last_became_active":"2024-03-16T08:16:05.654297+0000","last_became_peered":"2024-03-16T08:16:05.654297+0000","last_unstale":"2024-03-16T08:16:33.622106+0000","last_undegraded":"2024-03-16T08:16:33.622106+0000","last_fullsized":"2024-03-16T08:16:33.622106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:50.079111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386863+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:16:05.386863+0000","last_peered":"2024-03-16T08:16:05.386863+0000","last_clean":"2024-03-16T08:16:05.386863+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:16:05.386863+0000","last_undegraded":"2024-03-16T08:16:05.386863+0000","last_fullsized":"2024-03-16T08:16:05.386863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635060+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:16:33.635060+0000","last_peered":"2024-03-16T08:16:33.635060+0000","last_clean":"2024-03-16T08:16:33.635060+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:16:33.635060+0000","last_undegraded":"2024-03-16T08:16:33.635060+0000","last_fullsized":"2024-03-16T08:16:33.635060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388254+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:16:05.388254+0000","last_peered":"2024-03-16T08:16:05.388254+0000","last_clean":"2024-03-16T08:16:05.388254+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:16:05.388254+0000","last_undegraded":"2024-03-16T08:16:05.388254+0000","last_fullsized":"2024-03-16T08:16:05.388254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637196+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:16:33.637196+0000","last_peered":"2024-03-16T08:16:33.637196+0000","last_clean":"2024-03-16T08:16:33.637196+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:16:33.637196+0000","last_undegraded":"2024-03-16T08:16:33.637196+0000","last_fullsized":"2024-03-16T08:16:33.637196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388257+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:16:05.388257+0000","last_peered":"2024-03-16T08:16:05.388257+0000","last_clean":"2024-03-16T08:16:05.388257+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:16:05.388257+0000","last_undegraded":"2024-03-16T08:16:05.388257+0000","last_fullsized":"2024-03-16T08:16:05.388257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634147+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:16:33.634147+0000","last_peered":"2024-03-16T08:16:33.634147+0000","last_clean":"2024-03-16T08:16:33.634147+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:16:33.634147+0000","last_undegraded":"2024-03-16T08:16:33.634147+0000","last_fullsized":"2024-03-16T08:16:33.634147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.698827+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:16:59.698827+0000","last_peered":"2024-03-16T08:16:59.698827+0000","last_clean":"2024-03-16T08:16:59.698827+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:16:59.698827+0000","last_undegraded":"2024-03-16T08:16:59.698827+0000","last_fullsized":"2024-03-16T08:16:59.698827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616873+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:16:33.616873+0000","last_peered":"2024-03-16T08:16:33.616873+0000","last_clean":"2024-03-16T08:16:33.616873+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:16:33.616873+0000","last_undegraded":"2024-03-16T08:16:33.616873+0000","last_fullsized":"2024-03-16T08:16:33.616873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383442+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:16:05.383442+0000","last_peered":"2024-03-16T08:16:05.383442+0000","last_clean":"2024-03-16T08:16:05.383442+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:16:05.383442+0000","last_undegraded":"2024-03-16T08:16:05.383442+0000","last_fullsized":"2024-03-16T08:16:05.383442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634056+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:16:33.634056+0000","last_peered":"2024-03-16T08:16:33.634056+0000","last_clean":"2024-03-16T08:16:33.634056+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:16:33.634056+0000","last_undegraded":"2024-03-16T08:16:33.634056+0000","last_fullsized":"2024-03-16T08:16:33.634056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385161+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:16:05.385161+0000","last_peered":"2024-03-16T08:16:05.385161+0000","last_clean":"2024-03-16T08:16:05.385161+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:16:05.385161+0000","last_undegraded":"2024-03-16T08:16:05.385161+0000","last_fullsized":"2024-03-16T08:16:05.385161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624699+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:16:33.624699+0000","last_peered":"2024-03-16T08:16:33.624699+0000","last_clean":"2024-03-16T08:16:33.624699+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:16:33.624699+0000","last_undegraded":"2024-03-16T08:16:33.624699+0000","last_fullsized":"2024-03-16T08:16:33.624699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387954+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:16:05.387954+0000","last_peered":"2024-03-16T08:16:05.387954+0000","last_clean":"2024-03-16T08:16:05.387954+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:16:05.387954+0000","last_undegraded":"2024-03-16T08:16:05.387954+0000","last_fullsized":"2024-03-16T08:16:05.387954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625831+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:16:33.625831+0000","last_peered":"2024-03-16T08:16:33.625831+0000","last_clean":"2024-03-16T08:16:33.625831+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:16:33.625831+0000","last_undegraded":"2024-03-16T08:16:33.625831+0000","last_fullsized":"2024-03-16T08:16:33.625831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:55.693854+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:16:55.693854+0000","last_peered":"2024-03-16T08:16:55.693854+0000","last_clean":"2024-03-16T08:16:55.693854+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:16:55.693854+0000","last_undegraded":"2024-03-16T08:16:55.693854+0000","last_fullsized":"2024-03-16T08:16:55.693854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638002+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:16:33.638002+0000","last_peered":"2024-03-16T08:16:33.638002+0000","last_clean":"2024-03-16T08:16:33.638002+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:16:33.638002+0000","last_undegraded":"2024-03-16T08:16:33.638002+0000","last_fullsized":"2024-03-16T08:16:33.638002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381137+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:16:05.381137+0000","last_peered":"2024-03-16T08:16:05.381137+0000","last_clean":"2024-03-16T08:16:05.381137+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:16:05.381137+0000","last_undegraded":"2024-03-16T08:16:05.381137+0000","last_fullsized":"2024-03-16T08:16:05.381137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630292+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:16:33.630292+0000","last_peered":"2024-03-16T08:16:33.630292+0000","last_clean":"2024-03-16T08:16:33.630292+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:16:33.630292+0000","last_undegraded":"2024-03-16T08:16:33.630292+0000","last_fullsized":"2024-03-16T08:16:33.630292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389097+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:16:05.389097+0000","last_peered":"2024-03-16T08:16:05.389097+0000","last_clean":"2024-03-16T08:16:05.389097+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:16:05.389097+0000","last_undegraded":"2024-03-16T08:16:05.389097+0000","last_fullsized":"2024-03-16T08:16:05.389097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639262+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:16:33.639262+0000","last_peered":"2024-03-16T08:16:33.639262+0000","last_clean":"2024-03-16T08:16:33.639262+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:16:33.639262+0000","last_undegraded":"2024-03-16T08:16:33.639262+0000","last_fullsized":"2024-03-16T08:16:33.639262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":279,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:56.694997+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:16:56.694997+0000","last_peered":"2024-03-16T08:16:56.694997+0000","last_clean":"2024-03-16T08:16:56.694997+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:16:56.694997+0000","last_undegraded":"2024-03-16T08:16:56.694997+0000","last_fullsized":"2024-03-16T08:16:56.694997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":174,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629485+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:16:33.629485+0000","last_peered":"2024-03-16T08:16:33.629485+0000","last_clean":"2024-03-16T08:16:33.629485+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:16:33.629485+0000","last_undegraded":"2024-03-16T08:16:33.629485+0000","last_fullsized":"2024-03-16T08:16:33.629485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653483+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:16:05.653483+0000","last_peered":"2024-03-16T08:16:05.653483+0000","last_clean":"2024-03-16T08:16:05.653483+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:16:05.653483+0000","last_undegraded":"2024-03-16T08:16:05.653483+0000","last_fullsized":"2024-03-16T08:16:05.653483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629218+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:16:33.629218+0000","last_peered":"2024-03-16T08:16:33.629218+0000","last_clean":"2024-03-16T08:16:33.629218+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:16:33.629218+0000","last_undegraded":"2024-03-16T08:16:33.629218+0000","last_fullsized":"2024-03-16T08:16:33.629218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388595+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:16:05.388595+0000","last_peered":"2024-03-16T08:16:05.388595+0000","last_clean":"2024-03-16T08:16:05.388595+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:16:05.388595+0000","last_undegraded":"2024-03-16T08:16:05.388595+0000","last_fullsized":"2024-03-16T08:16:05.388595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628965+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:16:33.628965+0000","last_peered":"2024-03-16T08:16:33.628965+0000","last_clean":"2024-03-16T08:16:33.628965+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:16:33.628965+0000","last_undegraded":"2024-03-16T08:16:33.628965+0000","last_fullsized":"2024-03-16T08:16:33.628965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.716883+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:16:59.716883+0000","last_peered":"2024-03-16T08:16:59.716883+0000","last_clean":"2024-03-16T08:16:59.716883+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:16:59.716883+0000","last_undegraded":"2024-03-16T08:16:59.716883+0000","last_fullsized":"2024-03-16T08:16:59.716883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615961+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:16:33.615961+0000","last_peered":"2024-03-16T08:16:33.615961+0000","last_clean":"2024-03-16T08:16:33.615961+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:16:33.615961+0000","last_undegraded":"2024-03-16T08:16:33.615961+0000","last_fullsized":"2024-03-16T08:16:33.615961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387994+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:16:05.387994+0000","last_peered":"2024-03-16T08:16:05.387994+0000","last_clean":"2024-03-16T08:16:05.387994+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:16:05.387994+0000","last_undegraded":"2024-03-16T08:16:05.387994+0000","last_fullsized":"2024-03-16T08:16:05.387994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636796+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:16:33.636796+0000","last_peered":"2024-03-16T08:16:33.636796+0000","last_clean":"2024-03-16T08:16:33.636796+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:16:33.636796+0000","last_undegraded":"2024-03-16T08:16:33.636796+0000","last_fullsized":"2024-03-16T08:16:33.636796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.394865+0000","last_change":"2024-03-16T08:16:05.394457+0000","last_active":"2024-03-16T08:16:05.394865+0000","last_peered":"2024-03-16T08:16:05.394865+0000","last_clean":"2024-03-16T08:16:05.394865+0000","last_became_active":"2024-03-16T08:16:05.390239+0000","last_became_peered":"2024-03-16T08:16:05.390239+0000","last_unstale":"2024-03-16T08:16:05.394865+0000","last_undegraded":"2024-03-16T08:16:05.394865+0000","last_fullsized":"2024-03-16T08:16:05.394865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:29:29.442138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631505+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:16:33.631505+0000","last_peered":"2024-03-16T08:16:33.631505+0000","last_clean":"2024-03-16T08:16:33.631505+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:16:33.631505+0000","last_undegraded":"2024-03-16T08:16:33.631505+0000","last_fullsized":"2024-03-16T08:16:33.631505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.390744+0000","last_change":"2024-03-16T08:16:05.390744+0000","last_active":"2024-03-16T08:16:05.390744+0000","last_peered":"2024-03-16T08:16:05.390744+0000","last_clean":"2024-03-16T08:16:05.390744+0000","last_became_active":"2024-03-16T08:16:05.390378+0000","last_became_peered":"2024-03-16T08:16:05.390378+0000","last_unstale":"2024-03-16T08:16:05.390744+0000","last_undegraded":"2024-03-16T08:16:05.390744+0000","last_fullsized":"2024-03-16T08:16:05.390744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:13:29.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.617438+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:16:33.617438+0000","last_peered":"2024-03-16T08:16:33.617438+0000","last_clean":"2024-03-16T08:16:33.617438+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:16:33.617438+0000","last_undegraded":"2024-03-16T08:16:33.617438+0000","last_fullsized":"2024-03-16T08:16:33.617438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380895+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:16:05.380895+0000","last_peered":"2024-03-16T08:16:05.380895+0000","last_clean":"2024-03-16T08:16:05.380895+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:16:05.380895+0000","last_undegraded":"2024-03-16T08:16:05.380895+0000","last_fullsized":"2024-03-16T08:16:05.380895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.620223+0000","last_change":"2024-03-16T08:16:05.389020+0000","last_active":"2024-03-16T08:16:33.620223+0000","last_peered":"2024-03-16T08:16:33.620223+0000","last_clean":"2024-03-16T08:16:33.620223+0000","last_became_active":"2024-03-16T08:16:05.388668+0000","last_became_peered":"2024-03-16T08:16:05.388668+0000","last_unstale":"2024-03-16T08:16:33.620223+0000","last_undegraded":"2024-03-16T08:16:33.620223+0000","last_fullsized":"2024-03-16T08:16:33.620223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:34:04.661388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633962+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:16:33.633962+0000","last_peered":"2024-03-16T08:16:33.633962+0000","last_clean":"2024-03-16T08:16:33.633962+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:16:33.633962+0000","last_undegraded":"2024-03-16T08:16:33.633962+0000","last_fullsized":"2024-03-16T08:16:33.633962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":136,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:58.715510+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:16:58.715510+0000","last_peered":"2024-03-16T08:16:58.715510+0000","last_clean":"2024-03-16T08:16:58.715510+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:16:58.715510+0000","last_undegraded":"2024-03-16T08:16:58.715510+0000","last_fullsized":"2024-03-16T08:16:58.715510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636330+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:16:33.636330+0000","last_peered":"2024-03-16T08:16:33.636330+0000","last_clean":"2024-03-16T08:16:33.636330+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:16:33.636330+0000","last_undegraded":"2024-03-16T08:16:33.636330+0000","last_fullsized":"2024-03-16T08:16:33.636330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387927+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:16:05.387927+0000","last_peered":"2024-03-16T08:16:05.387927+0000","last_clean":"2024-03-16T08:16:05.387927+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:16:05.387927+0000","last_undegraded":"2024-03-16T08:16:05.387927+0000","last_fullsized":"2024-03-16T08:16:05.387927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380865+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:16:05.380865+0000","last_peered":"2024-03-16T08:16:05.380865+0000","last_clean":"2024-03-16T08:16:05.380865+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:16:05.380865+0000","last_undegraded":"2024-03-16T08:16:05.380865+0000","last_fullsized":"2024-03-16T08:16:05.380865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.627112+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:16:33.627112+0000","last_peered":"2024-03-16T08:16:33.627112+0000","last_clean":"2024-03-16T08:16:33.627112+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:16:33.627112+0000","last_undegraded":"2024-03-16T08:16:33.627112+0000","last_fullsized":"2024-03-16T08:16:33.627112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380918+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:16:05.380918+0000","last_peered":"2024-03-16T08:16:05.380918+0000","last_clean":"2024-03-16T08:16:05.380918+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:16:05.380918+0000","last_undegraded":"2024-03-16T08:16:05.380918+0000","last_fullsized":"2024-03-16T08:16:05.380918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388079+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:16:05.388079+0000","last_peered":"2024-03-16T08:16:05.388079+0000","last_clean":"2024-03-16T08:16:05.388079+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:16:05.388079+0000","last_undegraded":"2024-03-16T08:16:05.388079+0000","last_fullsized":"2024-03-16T08:16:05.388079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616704+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:16:33.616704+0000","last_peered":"2024-03-16T08:16:33.616704+0000","last_clean":"2024-03-16T08:16:33.616704+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:16:33.616704+0000","last_undegraded":"2024-03-16T08:16:33.616704+0000","last_fullsized":"2024-03-16T08:16:33.616704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380845+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:16:05.380845+0000","last_peered":"2024-03-16T08:16:05.380845+0000","last_clean":"2024-03-16T08:16:05.380845+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:16:05.380845+0000","last_undegraded":"2024-03-16T08:16:05.380845+0000","last_fullsized":"2024-03-16T08:16:05.380845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624878+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:16:33.624878+0000","last_peered":"2024-03-16T08:16:33.624878+0000","last_clean":"2024-03-16T08:16:33.624878+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:16:33.624878+0000","last_undegraded":"2024-03-16T08:16:33.624878+0000","last_fullsized":"2024-03-16T08:16:33.624878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388054+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:16:05.388054+0000","last_peered":"2024-03-16T08:16:05.388054+0000","last_clean":"2024-03-16T08:16:05.388054+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:16:05.388054+0000","last_undegraded":"2024-03-16T08:16:05.388054+0000","last_fullsized":"2024-03-16T08:16:05.388054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632266+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:16:33.632266+0000","last_peered":"2024-03-16T08:16:33.632266+0000","last_clean":"2024-03-16T08:16:33.632266+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:16:33.632266+0000","last_undegraded":"2024-03-16T08:16:33.632266+0000","last_fullsized":"2024-03-16T08:16:33.632266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652991+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:16:05.652991+0000","last_peered":"2024-03-16T08:16:05.652991+0000","last_clean":"2024-03-16T08:16:05.652991+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:16:05.652991+0000","last_undegraded":"2024-03-16T08:16:05.652991+0000","last_fullsized":"2024-03-16T08:16:05.652991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.585204+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:16:33.585204+0000","last_peered":"2024-03-16T08:16:33.585204+0000","last_clean":"2024-03-16T08:16:33.585204+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:16:33.585204+0000","last_undegraded":"2024-03-16T08:16:33.585204+0000","last_fullsized":"2024-03-16T08:16:33.585204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384150+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:16:05.384150+0000","last_peered":"2024-03-16T08:16:05.384150+0000","last_clean":"2024-03-16T08:16:05.384150+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:16:05.384150+0000","last_undegraded":"2024-03-16T08:16:05.384150+0000","last_fullsized":"2024-03-16T08:16:05.384150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625869+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:16:33.625869+0000","last_peered":"2024-03-16T08:16:33.625869+0000","last_clean":"2024-03-16T08:16:33.625869+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:16:33.625869+0000","last_undegraded":"2024-03-16T08:16:33.625869+0000","last_fullsized":"2024-03-16T08:16:33.625869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653334+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:16:05.653334+0000","last_peered":"2024-03-16T08:16:05.653334+0000","last_clean":"2024-03-16T08:16:05.653334+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:16:05.653334+0000","last_undegraded":"2024-03-16T08:16:05.653334+0000","last_fullsized":"2024-03-16T08:16:05.653334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628250+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:16:33.628250+0000","last_peered":"2024-03-16T08:16:33.628250+0000","last_clean":"2024-03-16T08:16:33.628250+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:16:33.628250+0000","last_undegraded":"2024-03-16T08:16:33.628250+0000","last_fullsized":"2024-03-16T08:16:33.628250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387802+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:16:05.387802+0000","last_peered":"2024-03-16T08:16:05.387802+0000","last_clean":"2024-03-16T08:16:05.387802+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:16:05.387802+0000","last_undegraded":"2024-03-16T08:16:05.387802+0000","last_fullsized":"2024-03-16T08:16:05.387802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637947+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:16:33.637947+0000","last_peered":"2024-03-16T08:16:33.637947+0000","last_clean":"2024-03-16T08:16:33.637947+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:16:33.637947+0000","last_undegraded":"2024-03-16T08:16:33.637947+0000","last_fullsized":"2024-03-16T08:16:33.637947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653465+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:16:05.653465+0000","last_peered":"2024-03-16T08:16:05.653465+0000","last_clean":"2024-03-16T08:16:05.653465+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:16:05.653465+0000","last_undegraded":"2024-03-16T08:16:05.653465+0000","last_fullsized":"2024-03-16T08:16:05.653465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638034+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:16:33.638034+0000","last_peered":"2024-03-16T08:16:33.638034+0000","last_clean":"2024-03-16T08:16:33.638034+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:16:33.638034+0000","last_undegraded":"2024-03-16T08:16:33.638034+0000","last_fullsized":"2024-03-16T08:16:33.638034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388862+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:16:05.388862+0000","last_peered":"2024-03-16T08:16:05.388862+0000","last_clean":"2024-03-16T08:16:05.388862+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:16:05.388862+0000","last_undegraded":"2024-03-16T08:16:05.388862+0000","last_fullsized":"2024-03-16T08:16:05.388862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.619455+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:16:33.619455+0000","last_peered":"2024-03-16T08:16:33.619455+0000","last_clean":"2024-03-16T08:16:33.619455+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:16:33.619455+0000","last_undegraded":"2024-03-16T08:16:33.619455+0000","last_fullsized":"2024-03-16T08:16:33.619455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384729+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:16:05.384729+0000","last_peered":"2024-03-16T08:16:05.384729+0000","last_clean":"2024-03-16T08:16:05.384729+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:16:05.384729+0000","last_undegraded":"2024-03-16T08:16:05.384729+0000","last_fullsized":"2024-03-16T08:16:05.384729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630956+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:16:33.630956+0000","last_peered":"2024-03-16T08:16:33.630956+0000","last_clean":"2024-03-16T08:16:33.630956+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:16:33.630956+0000","last_undegraded":"2024-03-16T08:16:33.630956+0000","last_fullsized":"2024-03-16T08:16:33.630956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624759+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:16:33.624759+0000","last_peered":"2024-03-16T08:16:33.624759+0000","last_clean":"2024-03-16T08:16:33.624759+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:16:33.624759+0000","last_undegraded":"2024-03-16T08:16:33.624759+0000","last_fullsized":"2024-03-16T08:16:33.624759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631321+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:16:33.631321+0000","last_peered":"2024-03-16T08:16:33.631321+0000","last_clean":"2024-03-16T08:16:33.631321+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:16:33.631321+0000","last_undegraded":"2024-03-16T08:16:33.631321+0000","last_fullsized":"2024-03-16T08:16:33.631321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622948+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:16:33.622948+0000","last_peered":"2024-03-16T08:16:33.622948+0000","last_clean":"2024-03-16T08:16:33.622948+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:16:33.622948+0000","last_undegraded":"2024-03-16T08:16:33.622948+0000","last_fullsized":"2024-03-16T08:16:33.622948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633266+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:16:33.633266+0000","last_peered":"2024-03-16T08:16:33.633266+0000","last_clean":"2024-03-16T08:16:33.633266+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:16:33.633266+0000","last_undegraded":"2024-03-16T08:16:33.633266+0000","last_fullsized":"2024-03-16T08:16:33.633266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632885+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:16:33.632885+0000","last_peered":"2024-03-16T08:16:33.632885+0000","last_clean":"2024-03-16T08:16:33.632885+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:16:33.632885+0000","last_undegraded":"2024-03-16T08:16:33.632885+0000","last_fullsized":"2024-03-16T08:16:33.632885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623433+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:16:33.623433+0000","last_peered":"2024-03-16T08:16:33.623433+0000","last_clean":"2024-03-16T08:16:33.623433+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:16:33.623433+0000","last_undegraded":"2024-03-16T08:16:33.623433+0000","last_fullsized":"2024-03-16T08:16:33.623433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639557+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:16:33.639557+0000","last_peered":"2024-03-16T08:16:33.639557+0000","last_clean":"2024-03-16T08:16:33.639557+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:16:33.639557+0000","last_undegraded":"2024-03-16T08:16:33.639557+0000","last_fullsized":"2024-03-16T08:16:33.639557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616343+0000","last_change":"2024-03-16T08:16:05.654782+0000","last_active":"2024-03-16T08:16:33.616343+0000","last_peered":"2024-03-16T08:16:33.616343+0000","last_clean":"2024-03-16T08:16:33.616343+0000","last_became_active":"2024-03-16T08:16:05.654117+0000","last_became_peered":"2024-03-16T08:16:05.654117+0000","last_unstale":"2024-03-16T08:16:33.616343+0000","last_undegraded":"2024-03-16T08:16:33.616343+0000","last_fullsized":"2024-03-16T08:16:33.616343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:17:45.678541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.614995+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:16:33.614995+0000","last_peered":"2024-03-16T08:16:33.614995+0000","last_clean":"2024-03-16T08:16:33.614995+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:16:33.614995+0000","last_undegraded":"2024-03-16T08:16:33.614995+0000","last_fullsized":"2024-03-16T08:16:33.614995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624368+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:16:33.624368+0000","last_peered":"2024-03-16T08:16:33.624368+0000","last_clean":"2024-03-16T08:16:33.624368+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:16:33.624368+0000","last_undegraded":"2024-03-16T08:16:33.624368+0000","last_fullsized":"2024-03-16T08:16:33.624368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639654+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:16:33.639654+0000","last_peered":"2024-03-16T08:16:33.639654+0000","last_clean":"2024-03-16T08:16:33.639654+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:16:33.639654+0000","last_undegraded":"2024-03-16T08:16:33.639654+0000","last_fullsized":"2024-03-16T08:16:33.639654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629430+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:16:33.629430+0000","last_peered":"2024-03-16T08:16:33.629430+0000","last_clean":"2024-03-16T08:16:33.629430+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:16:33.629430+0000","last_undegraded":"2024-03-16T08:16:33.629430+0000","last_fullsized":"2024-03-16T08:16:33.629430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633659+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:16:33.633659+0000","last_peered":"2024-03-16T08:16:33.633659+0000","last_clean":"2024-03-16T08:16:33.633659+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:16:33.633659+0000","last_undegraded":"2024-03-16T08:16:33.633659+0000","last_fullsized":"2024-03-16T08:16:33.633659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65437,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":416,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196619,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":459215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1111969,"data_compressed":37963,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051420,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":520214,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521788,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1051104,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61603,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63177,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62873,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:17:02.066 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-16T08:17:02.066 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T08:17:02.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 -- 172.21.15.49:0/95663452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4107560 msgr2=0x7ff3d41099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/95663452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4107560 0x7ff3d41099b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009ca0 tx=0x7ff3cc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 -- 172.21.15.49:0/95663452 shutdown_connections 2024-03-16T08:17:02.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/95663452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4107560 0x7ff3d41099b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/95663452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4104c30 0x7ff3d4107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/95663452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4102300 0x7ff3d41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 -- 172.21.15.49:0/95663452 >> 172.21.15.49:0/95663452 conn(0x7ff3d40fbd10 msgr2=0x7ff3d40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 -- 172.21.15.49:0/95663452 shutdown_connections 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 -- 172.21.15.49:0/95663452 wait complete. 2024-03-16T08:17:02.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.214+0000 7ff3d93cd640 1 Processor -- start 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 -- start start 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4102300 0x7ff3d4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4107560 0x7ff3d419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d410d4b0 con 0x7ff3d4104c30 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d410d330 con 0x7ff3d4102300 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d93cd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d410d630 con 0x7ff3d4107560 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60486/0 (socket says 172.21.15.49:60486) 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 -- 172.21.15.49:0/2399550069 learned_addr learned my addr 172.21.15.49:0/2399550069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:02.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d37fe640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4107560 0x7ff3d419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4102300 0x7ff3d4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 -- 172.21.15.49:0/2399550069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4107560 msgr2=0x7ff3d419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4107560 0x7ff3d419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 -- 172.21.15.49:0/2399550069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4102300 msgr2=0x7ff3d4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4102300 0x7ff3d4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 -- 172.21.15.49:0/2399550069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d419f6d0 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3d27fc640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400e9c0 tx=0x7ff3c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.215+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400cd80 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.216+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4005500 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.216+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d419f9c0 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.216+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4010690 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.216+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d419ff10 con 0x7ff3d4104c30 2024-03-16T08:17:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.217+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d40fd800 con 0x7ff3d4104c30 2024-03-16T08:17:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.218+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff3c40046c0 con 0x7ff3d4104c30 2024-03-16T08:17:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.219+0000 7ff3cbfff640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 0x7ff3c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.219+0000 7ff3d2ffd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 0x7ff3c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.219+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff3c40ced90 con 0x7ff3d4104c30 2024-03-16T08:17:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.219+0000 7ff3d2ffd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 0x7ff3c00a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009bd0 tx=0x7ff3bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.223+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3c409be60 con 0x7ff3d4104c30 2024-03-16T08:17:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.404+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3d4061c20 con 0x7ff3c00a2800 2024-03-16T08:17:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.411+0000 7ff3cbfff640 1 -- 172.21.15.49:0/2399550069 <== mgr.14152 v2:172.21.15.49:6800/2256169095 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259048 (secure 0 0 0) 0x7ff3d4061c20 con 0x7ff3c00a2800 2024-03-16T08:17:02.414 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:02.415 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-16T08:17:02.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 msgr2=0x7ff3c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 0x7ff3c00a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009bd0 tx=0x7ff3bc005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 msgr2=0x7ff3d4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400e9c0 tx=0x7ff3c400ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 shutdown_connections 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3d4107560 0x7ff3d419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff3c00a2800 0x7ff3c00a4cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d4104c30 0x7ff3d4197ff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 --2- 172.21.15.49:0/2399550069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d4102300 0x7ff3d4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 >> 172.21.15.49:0/2399550069 conn(0x7ff3d40fbd10 msgr2=0x7ff3d4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.414+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 shutdown_connections 2024-03-16T08:17:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.415+0000 7ff3d93cd640 1 -- 172.21.15.49:0/2399550069 wait complete. 2024-03-16T08:17:02.426 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-03-16T08:17:01.609234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656039,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":835,"num_read_kb":535,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":780,"ondisk_log_size":780,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356644,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585180,"statfs":{"total":1151940427776,"available":1151575224320,"internally_reserved":0,"allocated":11931648,"data_stored":7471757,"data_compressed":113889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002476"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637038+0000","last_change":"2024-03-16T08:15:02.175257+0000","last_active":"2024-03-16T08:16:33.637038+0000","last_peered":"2024-03-16T08:16:33.637038+0000","last_clean":"2024-03-16T08:16:33.637038+0000","last_became_active":"2024-03-16T08:15:02.172785+0000","last_became_peered":"2024-03-16T08:15:02.172785+0000","last_unstale":"2024-03-16T08:16:33.637038+0000","last_undegraded":"2024-03-16T08:16:33.637038+0000","last_fullsized":"2024-03-16T08:16:33.637038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:07:44.591373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.626061+0000","last_change":"2024-03-16T08:15:02.179893+0000","last_active":"2024-03-16T08:16:33.626061+0000","last_peered":"2024-03-16T08:16:33.626061+0000","last_clean":"2024-03-16T08:16:33.626061+0000","last_became_active":"2024-03-16T08:15:02.179545+0000","last_became_peered":"2024-03-16T08:15:02.179545+0000","last_unstale":"2024-03-16T08:16:33.626061+0000","last_undegraded":"2024-03-16T08:16:33.626061+0000","last_fullsized":"2024-03-16T08:16:33.626061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:33.871327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635069+0000","last_change":"2024-03-16T08:15:02.172613+0000","last_active":"2024-03-16T08:16:33.635069+0000","last_peered":"2024-03-16T08:16:33.635069+0000","last_clean":"2024-03-16T08:16:33.635069+0000","last_became_active":"2024-03-16T08:15:02.172245+0000","last_became_peered":"2024-03-16T08:15:02.172245+0000","last_unstale":"2024-03-16T08:16:33.635069+0000","last_undegraded":"2024-03-16T08:16:33.635069+0000","last_fullsized":"2024-03-16T08:16:33.635069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:40.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634668+0000","last_change":"2024-03-16T08:15:02.170418+0000","last_active":"2024-03-16T08:16:33.634668+0000","last_peered":"2024-03-16T08:16:33.634668+0000","last_clean":"2024-03-16T08:16:33.634668+0000","last_became_active":"2024-03-16T08:15:02.169770+0000","last_became_peered":"2024-03-16T08:15:02.169770+0000","last_unstale":"2024-03-16T08:16:33.634668+0000","last_undegraded":"2024-03-16T08:16:33.634668+0000","last_fullsized":"2024-03-16T08:16:33.634668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:04.580448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616273+0000","last_change":"2024-03-16T08:15:02.173479+0000","last_active":"2024-03-16T08:16:33.616273+0000","last_peered":"2024-03-16T08:16:33.616273+0000","last_clean":"2024-03-16T08:16:33.616273+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.616273+0000","last_undegraded":"2024-03-16T08:16:33.616273+0000","last_fullsized":"2024-03-16T08:16:33.616273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:08.746465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631271+0000","last_change":"2024-03-16T08:15:02.169256+0000","last_active":"2024-03-16T08:16:33.631271+0000","last_peered":"2024-03-16T08:16:33.631271+0000","last_clean":"2024-03-16T08:16:33.631271+0000","last_became_active":"2024-03-16T08:15:02.168936+0000","last_became_peered":"2024-03-16T08:15:02.168936+0000","last_unstale":"2024-03-16T08:16:33.631271+0000","last_undegraded":"2024-03-16T08:16:33.631271+0000","last_fullsized":"2024-03-16T08:16:33.631271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:46:25.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622976+0000","last_change":"2024-03-16T08:15:02.173245+0000","last_active":"2024-03-16T08:16:33.622976+0000","last_peered":"2024-03-16T08:16:33.622976+0000","last_clean":"2024-03-16T08:16:33.622976+0000","last_became_active":"2024-03-16T08:15:02.172852+0000","last_became_peered":"2024-03-16T08:15:02.172852+0000","last_unstale":"2024-03-16T08:16:33.622976+0000","last_undegraded":"2024-03-16T08:16:33.622976+0000","last_fullsized":"2024-03-16T08:16:33.622976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:12.236295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621910+0000","last_change":"2024-03-16T08:15:02.177475+0000","last_active":"2024-03-16T08:16:33.621910+0000","last_peered":"2024-03-16T08:16:33.621910+0000","last_clean":"2024-03-16T08:16:33.621910+0000","last_became_active":"2024-03-16T08:15:02.177013+0000","last_became_peered":"2024-03-16T08:15:02.177013+0000","last_unstale":"2024-03-16T08:16:33.621910+0000","last_undegraded":"2024-03-16T08:16:33.621910+0000","last_fullsized":"2024-03-16T08:16:33.621910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:35:07.963606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631822+0000","last_change":"2024-03-16T08:15:02.171003+0000","last_active":"2024-03-16T08:16:33.631822+0000","last_peered":"2024-03-16T08:16:33.631822+0000","last_clean":"2024-03-16T08:16:33.631822+0000","last_became_active":"2024-03-16T08:15:02.170600+0000","last_became_peered":"2024-03-16T08:15:02.170600+0000","last_unstale":"2024-03-16T08:16:33.631822+0000","last_undegraded":"2024-03-16T08:16:33.631822+0000","last_fullsized":"2024-03-16T08:16:33.631822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:50.255949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631465+0000","last_change":"2024-03-16T08:15:02.173851+0000","last_active":"2024-03-16T08:16:33.631465+0000","last_peered":"2024-03-16T08:16:33.631465+0000","last_clean":"2024-03-16T08:16:33.631465+0000","last_became_active":"2024-03-16T08:15:02.173514+0000","last_became_peered":"2024-03-16T08:15:02.173514+0000","last_unstale":"2024-03-16T08:16:33.631465+0000","last_undegraded":"2024-03-16T08:16:33.631465+0000","last_fullsized":"2024-03-16T08:16:33.631465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:08:56.892458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637825+0000","last_change":"2024-03-16T08:15:02.171360+0000","last_active":"2024-03-16T08:16:33.637825+0000","last_peered":"2024-03-16T08:16:33.637825+0000","last_clean":"2024-03-16T08:16:33.637825+0000","last_became_active":"2024-03-16T08:15:02.170671+0000","last_became_peered":"2024-03-16T08:15:02.170671+0000","last_unstale":"2024-03-16T08:16:33.637825+0000","last_undegraded":"2024-03-16T08:16:33.637825+0000","last_fullsized":"2024-03-16T08:16:33.637825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:39.910885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634424+0000","last_change":"2024-03-16T08:15:02.173451+0000","last_active":"2024-03-16T08:16:33.634424+0000","last_peered":"2024-03-16T08:16:33.634424+0000","last_clean":"2024-03-16T08:16:33.634424+0000","last_became_active":"2024-03-16T08:15:02.172465+0000","last_became_peered":"2024-03-16T08:15:02.172465+0000","last_unstale":"2024-03-16T08:16:33.634424+0000","last_undegraded":"2024-03-16T08:16:33.634424+0000","last_fullsized":"2024-03-16T08:16:33.634424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:03:48.264306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615731+0000","last_change":"2024-03-16T08:16:05.389606+0000","last_active":"2024-03-16T08:16:33.615731+0000","last_peered":"2024-03-16T08:16:33.615731+0000","last_clean":"2024-03-16T08:16:33.615731+0000","last_became_active":"2024-03-16T08:16:05.389062+0000","last_became_peered":"2024-03-16T08:16:05.389062+0000","last_unstale":"2024-03-16T08:16:33.615731+0000","last_undegraded":"2024-03-16T08:16:33.615731+0000","last_fullsized":"2024-03-16T08:16:33.615731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:04:08.093332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623216+0000","last_change":"2024-03-16T08:15:02.178800+0000","last_active":"2024-03-16T08:16:33.623216+0000","last_peered":"2024-03-16T08:16:33.623216+0000","last_clean":"2024-03-16T08:16:33.623216+0000","last_became_active":"2024-03-16T08:15:02.178475+0000","last_became_peered":"2024-03-16T08:15:02.178475+0000","last_unstale":"2024-03-16T08:16:33.623216+0000","last_undegraded":"2024-03-16T08:16:33.623216+0000","last_fullsized":"2024-03-16T08:16:33.623216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:25:23.508612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632440+0000","last_change":"2024-03-16T08:16:05.654998+0000","last_active":"2024-03-16T08:16:33.632440+0000","last_peered":"2024-03-16T08:16:33.632440+0000","last_clean":"2024-03-16T08:16:33.632440+0000","last_became_active":"2024-03-16T08:16:05.654444+0000","last_became_peered":"2024-03-16T08:16:05.654444+0000","last_unstale":"2024-03-16T08:16:33.632440+0000","last_undegraded":"2024-03-16T08:16:33.632440+0000","last_fullsized":"2024-03-16T08:16:33.632440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:04.343402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637327+0000","last_change":"2024-03-16T08:15:02.168586+0000","last_active":"2024-03-16T08:16:33.637327+0000","last_peered":"2024-03-16T08:16:33.637327+0000","last_clean":"2024-03-16T08:16:33.637327+0000","last_became_active":"2024-03-16T08:15:02.166938+0000","last_became_peered":"2024-03-16T08:15:02.166938+0000","last_unstale":"2024-03-16T08:16:33.637327+0000","last_undegraded":"2024-03-16T08:16:33.637327+0000","last_fullsized":"2024-03-16T08:16:33.637327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:38:53.873876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637250+0000","last_change":"2024-03-16T08:15:02.175675+0000","last_active":"2024-03-16T08:16:33.637250+0000","last_peered":"2024-03-16T08:16:33.637250+0000","last_clean":"2024-03-16T08:16:33.637250+0000","last_became_active":"2024-03-16T08:15:02.173187+0000","last_became_peered":"2024-03-16T08:15:02.173187+0000","last_unstale":"2024-03-16T08:16:33.637250+0000","last_undegraded":"2024-03-16T08:16:33.637250+0000","last_fullsized":"2024-03-16T08:16:33.637250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:22:38.912229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639565+0000","last_change":"2024-03-16T08:15:02.172794+0000","last_active":"2024-03-16T08:16:33.639565+0000","last_peered":"2024-03-16T08:16:33.639565+0000","last_clean":"2024-03-16T08:16:33.639565+0000","last_became_active":"2024-03-16T08:15:02.172434+0000","last_became_peered":"2024-03-16T08:15:02.172434+0000","last_unstale":"2024-03-16T08:16:33.639565+0000","last_undegraded":"2024-03-16T08:16:33.639565+0000","last_fullsized":"2024-03-16T08:16:33.639565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:47.678062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.621658+0000","last_change":"2024-03-16T08:15:02.166129+0000","last_active":"2024-03-16T08:16:33.621658+0000","last_peered":"2024-03-16T08:16:33.621658+0000","last_clean":"2024-03-16T08:16:33.621658+0000","last_became_active":"2024-03-16T08:15:02.165686+0000","last_became_peered":"2024-03-16T08:15:02.165686+0000","last_unstale":"2024-03-16T08:16:33.621658+0000","last_undegraded":"2024-03-16T08:16:33.621658+0000","last_fullsized":"2024-03-16T08:16:33.621658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:16:13.811506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380775+0000","last_change":"2024-03-16T08:15:01.166713+0000","last_active":"2024-03-16T08:16:05.380775+0000","last_peered":"2024-03-16T08:16:05.380775+0000","last_clean":"2024-03-16T08:16:05.380775+0000","last_became_active":"2024-03-16T08:15:01.163388+0000","last_became_peered":"2024-03-16T08:15:01.163388+0000","last_unstale":"2024-03-16T08:16:05.380775+0000","last_undegraded":"2024-03-16T08:16:05.380775+0000","last_fullsized":"2024-03-16T08:16:05.380775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:10.631436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622106+0000","last_change":"2024-03-16T08:16:05.654774+0000","last_active":"2024-03-16T08:16:33.622106+0000","last_peered":"2024-03-16T08:16:33.622106+0000","last_clean":"2024-03-16T08:16:33.622106+0000","last_became_active":"2024-03-16T08:16:05.654297+0000","last_became_peered":"2024-03-16T08:16:05.654297+0000","last_unstale":"2024-03-16T08:16:33.622106+0000","last_undegraded":"2024-03-16T08:16:33.622106+0000","last_fullsized":"2024-03-16T08:16:33.622106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:50.079111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.386863+0000","last_change":"2024-03-16T08:15:01.168748+0000","last_active":"2024-03-16T08:16:05.386863+0000","last_peered":"2024-03-16T08:16:05.386863+0000","last_clean":"2024-03-16T08:16:05.386863+0000","last_became_active":"2024-03-16T08:15:01.168423+0000","last_became_peered":"2024-03-16T08:15:01.168423+0000","last_unstale":"2024-03-16T08:16:05.386863+0000","last_undegraded":"2024-03-16T08:16:05.386863+0000","last_fullsized":"2024-03-16T08:16:05.386863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:24.430444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.635060+0000","last_change":"2024-03-16T08:15:02.172704+0000","last_active":"2024-03-16T08:16:33.635060+0000","last_peered":"2024-03-16T08:16:33.635060+0000","last_clean":"2024-03-16T08:16:33.635060+0000","last_became_active":"2024-03-16T08:15:02.171855+0000","last_became_peered":"2024-03-16T08:15:02.171855+0000","last_unstale":"2024-03-16T08:16:33.635060+0000","last_undegraded":"2024-03-16T08:16:33.635060+0000","last_fullsized":"2024-03-16T08:16:33.635060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:56.264620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388254+0000","last_change":"2024-03-16T08:15:01.170578+0000","last_active":"2024-03-16T08:16:05.388254+0000","last_peered":"2024-03-16T08:16:05.388254+0000","last_clean":"2024-03-16T08:16:05.388254+0000","last_became_active":"2024-03-16T08:15:01.169966+0000","last_became_peered":"2024-03-16T08:15:01.169966+0000","last_unstale":"2024-03-16T08:16:05.388254+0000","last_undegraded":"2024-03-16T08:16:05.388254+0000","last_fullsized":"2024-03-16T08:16:05.388254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:37:14.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637196+0000","last_change":"2024-03-16T08:15:02.174855+0000","last_active":"2024-03-16T08:16:33.637196+0000","last_peered":"2024-03-16T08:16:33.637196+0000","last_clean":"2024-03-16T08:16:33.637196+0000","last_became_active":"2024-03-16T08:15:02.174536+0000","last_became_peered":"2024-03-16T08:15:02.174536+0000","last_unstale":"2024-03-16T08:16:33.637196+0000","last_undegraded":"2024-03-16T08:16:33.637196+0000","last_fullsized":"2024-03-16T08:16:33.637196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:47.372489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388257+0000","last_change":"2024-03-16T08:15:01.167352+0000","last_active":"2024-03-16T08:16:05.388257+0000","last_peered":"2024-03-16T08:16:05.388257+0000","last_clean":"2024-03-16T08:16:05.388257+0000","last_became_active":"2024-03-16T08:15:01.167051+0000","last_became_peered":"2024-03-16T08:15:01.167051+0000","last_unstale":"2024-03-16T08:16:05.388257+0000","last_undegraded":"2024-03-16T08:16:05.388257+0000","last_fullsized":"2024-03-16T08:16:05.388257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:26:18.439192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634147+0000","last_change":"2024-03-16T08:15:02.175076+0000","last_active":"2024-03-16T08:16:33.634147+0000","last_peered":"2024-03-16T08:16:33.634147+0000","last_clean":"2024-03-16T08:16:33.634147+0000","last_became_active":"2024-03-16T08:15:02.171824+0000","last_became_peered":"2024-03-16T08:15:02.171824+0000","last_unstale":"2024-03-16T08:16:33.634147+0000","last_undegraded":"2024-03-16T08:16:33.634147+0000","last_fullsized":"2024-03-16T08:16:33.634147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:40.084697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.698827+0000","last_change":"2024-03-16T08:15:01.167553+0000","last_active":"2024-03-16T08:16:59.698827+0000","last_peered":"2024-03-16T08:16:59.698827+0000","last_clean":"2024-03-16T08:16:59.698827+0000","last_became_active":"2024-03-16T08:15:01.167240+0000","last_became_peered":"2024-03-16T08:15:01.167240+0000","last_unstale":"2024-03-16T08:16:59.698827+0000","last_undegraded":"2024-03-16T08:16:59.698827+0000","last_fullsized":"2024-03-16T08:16:59.698827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:44:14.079050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616873+0000","last_change":"2024-03-16T08:15:02.170256+0000","last_active":"2024-03-16T08:16:33.616873+0000","last_peered":"2024-03-16T08:16:33.616873+0000","last_clean":"2024-03-16T08:16:33.616873+0000","last_became_active":"2024-03-16T08:15:02.169716+0000","last_became_peered":"2024-03-16T08:15:02.169716+0000","last_unstale":"2024-03-16T08:16:33.616873+0000","last_undegraded":"2024-03-16T08:16:33.616873+0000","last_fullsized":"2024-03-16T08:16:33.616873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:05:10.065505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.383442+0000","last_change":"2024-03-16T08:15:01.168070+0000","last_active":"2024-03-16T08:16:05.383442+0000","last_peered":"2024-03-16T08:16:05.383442+0000","last_clean":"2024-03-16T08:16:05.383442+0000","last_became_active":"2024-03-16T08:15:01.167702+0000","last_became_peered":"2024-03-16T08:15:01.167702+0000","last_unstale":"2024-03-16T08:16:05.383442+0000","last_undegraded":"2024-03-16T08:16:05.383442+0000","last_fullsized":"2024-03-16T08:16:05.383442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:31:09.168246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.634056+0000","last_change":"2024-03-16T08:15:02.168302+0000","last_active":"2024-03-16T08:16:33.634056+0000","last_peered":"2024-03-16T08:16:33.634056+0000","last_clean":"2024-03-16T08:16:33.634056+0000","last_became_active":"2024-03-16T08:15:02.167933+0000","last_became_peered":"2024-03-16T08:15:02.167933+0000","last_unstale":"2024-03-16T08:16:33.634056+0000","last_undegraded":"2024-03-16T08:16:33.634056+0000","last_fullsized":"2024-03-16T08:16:33.634056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:59:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.385161+0000","last_change":"2024-03-16T08:15:01.168839+0000","last_active":"2024-03-16T08:16:05.385161+0000","last_peered":"2024-03-16T08:16:05.385161+0000","last_clean":"2024-03-16T08:16:05.385161+0000","last_became_active":"2024-03-16T08:15:01.168454+0000","last_became_peered":"2024-03-16T08:15:01.168454+0000","last_unstale":"2024-03-16T08:16:05.385161+0000","last_undegraded":"2024-03-16T08:16:05.385161+0000","last_fullsized":"2024-03-16T08:16:05.385161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:21:34.105854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624699+0000","last_change":"2024-03-16T08:15:02.177470+0000","last_active":"2024-03-16T08:16:33.624699+0000","last_peered":"2024-03-16T08:16:33.624699+0000","last_clean":"2024-03-16T08:16:33.624699+0000","last_became_active":"2024-03-16T08:15:02.176490+0000","last_became_peered":"2024-03-16T08:15:02.176490+0000","last_unstale":"2024-03-16T08:16:33.624699+0000","last_undegraded":"2024-03-16T08:16:33.624699+0000","last_fullsized":"2024-03-16T08:16:33.624699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:19:30.788068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387954+0000","last_change":"2024-03-16T08:15:01.165469+0000","last_active":"2024-03-16T08:16:05.387954+0000","last_peered":"2024-03-16T08:16:05.387954+0000","last_clean":"2024-03-16T08:16:05.387954+0000","last_became_active":"2024-03-16T08:15:01.165141+0000","last_became_peered":"2024-03-16T08:15:01.165141+0000","last_unstale":"2024-03-16T08:16:05.387954+0000","last_undegraded":"2024-03-16T08:16:05.387954+0000","last_fullsized":"2024-03-16T08:16:05.387954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:57:56.567671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625831+0000","last_change":"2024-03-16T08:15:02.169052+0000","last_active":"2024-03-16T08:16:33.625831+0000","last_peered":"2024-03-16T08:16:33.625831+0000","last_clean":"2024-03-16T08:16:33.625831+0000","last_became_active":"2024-03-16T08:15:02.168732+0000","last_became_peered":"2024-03-16T08:15:02.168732+0000","last_unstale":"2024-03-16T08:16:33.625831+0000","last_undegraded":"2024-03-16T08:16:33.625831+0000","last_fullsized":"2024-03-16T08:16:33.625831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:21.543115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:55.693854+0000","last_change":"2024-03-16T08:15:01.170466+0000","last_active":"2024-03-16T08:16:55.693854+0000","last_peered":"2024-03-16T08:16:55.693854+0000","last_clean":"2024-03-16T08:16:55.693854+0000","last_became_active":"2024-03-16T08:15:01.170105+0000","last_became_peered":"2024-03-16T08:15:01.170105+0000","last_unstale":"2024-03-16T08:16:55.693854+0000","last_undegraded":"2024-03-16T08:16:55.693854+0000","last_fullsized":"2024-03-16T08:16:55.693854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:53:16.821535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638002+0000","last_change":"2024-03-16T08:15:02.174544+0000","last_active":"2024-03-16T08:16:33.638002+0000","last_peered":"2024-03-16T08:16:33.638002+0000","last_clean":"2024-03-16T08:16:33.638002+0000","last_became_active":"2024-03-16T08:15:02.171789+0000","last_became_peered":"2024-03-16T08:15:02.171789+0000","last_unstale":"2024-03-16T08:16:33.638002+0000","last_undegraded":"2024-03-16T08:16:33.638002+0000","last_fullsized":"2024-03-16T08:16:33.638002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:24:07.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.381137+0000","last_change":"2024-03-16T08:15:01.167338+0000","last_active":"2024-03-16T08:16:05.381137+0000","last_peered":"2024-03-16T08:16:05.381137+0000","last_clean":"2024-03-16T08:16:05.381137+0000","last_became_active":"2024-03-16T08:15:01.167043+0000","last_became_peered":"2024-03-16T08:15:01.167043+0000","last_unstale":"2024-03-16T08:16:05.381137+0000","last_undegraded":"2024-03-16T08:16:05.381137+0000","last_fullsized":"2024-03-16T08:16:05.381137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:01:54.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630292+0000","last_change":"2024-03-16T08:15:02.173024+0000","last_active":"2024-03-16T08:16:33.630292+0000","last_peered":"2024-03-16T08:16:33.630292+0000","last_clean":"2024-03-16T08:16:33.630292+0000","last_became_active":"2024-03-16T08:15:02.172339+0000","last_became_peered":"2024-03-16T08:15:02.172339+0000","last_unstale":"2024-03-16T08:16:33.630292+0000","last_undegraded":"2024-03-16T08:16:33.630292+0000","last_fullsized":"2024-03-16T08:16:33.630292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:09.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.389097+0000","last_change":"2024-03-16T08:15:01.168350+0000","last_active":"2024-03-16T08:16:05.389097+0000","last_peered":"2024-03-16T08:16:05.389097+0000","last_clean":"2024-03-16T08:16:05.389097+0000","last_became_active":"2024-03-16T08:15:01.167998+0000","last_became_peered":"2024-03-16T08:15:01.167998+0000","last_unstale":"2024-03-16T08:16:05.389097+0000","last_undegraded":"2024-03-16T08:16:05.389097+0000","last_fullsized":"2024-03-16T08:16:05.389097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:04.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639262+0000","last_change":"2024-03-16T08:15:02.166737+0000","last_active":"2024-03-16T08:16:33.639262+0000","last_peered":"2024-03-16T08:16:33.639262+0000","last_clean":"2024-03-16T08:16:33.639262+0000","last_became_active":"2024-03-16T08:15:02.165959+0000","last_became_peered":"2024-03-16T08:15:02.165959+0000","last_unstale":"2024-03-16T08:16:33.639262+0000","last_undegraded":"2024-03-16T08:16:33.639262+0000","last_fullsized":"2024-03-16T08:16:33.639262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:32.372204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":279,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:56.694997+0000","last_change":"2024-03-16T08:15:01.164797+0000","last_active":"2024-03-16T08:16:56.694997+0000","last_peered":"2024-03-16T08:16:56.694997+0000","last_clean":"2024-03-16T08:16:56.694997+0000","last_became_active":"2024-03-16T08:15:01.164380+0000","last_became_peered":"2024-03-16T08:15:01.164380+0000","last_unstale":"2024-03-16T08:16:56.694997+0000","last_undegraded":"2024-03-16T08:16:56.694997+0000","last_fullsized":"2024-03-16T08:16:56.694997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:54:37.990060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":174,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629485+0000","last_change":"2024-03-16T08:15:02.168823+0000","last_active":"2024-03-16T08:16:33.629485+0000","last_peered":"2024-03-16T08:16:33.629485+0000","last_clean":"2024-03-16T08:16:33.629485+0000","last_became_active":"2024-03-16T08:15:02.168145+0000","last_became_peered":"2024-03-16T08:15:02.168145+0000","last_unstale":"2024-03-16T08:16:33.629485+0000","last_undegraded":"2024-03-16T08:16:33.629485+0000","last_fullsized":"2024-03-16T08:16:33.629485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:09:49.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653483+0000","last_change":"2024-03-16T08:15:01.165338+0000","last_active":"2024-03-16T08:16:05.653483+0000","last_peered":"2024-03-16T08:16:05.653483+0000","last_clean":"2024-03-16T08:16:05.653483+0000","last_became_active":"2024-03-16T08:15:01.164479+0000","last_became_peered":"2024-03-16T08:15:01.164479+0000","last_unstale":"2024-03-16T08:16:05.653483+0000","last_undegraded":"2024-03-16T08:16:05.653483+0000","last_fullsized":"2024-03-16T08:16:05.653483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:56:56.977595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629218+0000","last_change":"2024-03-16T08:15:02.167118+0000","last_active":"2024-03-16T08:16:33.629218+0000","last_peered":"2024-03-16T08:16:33.629218+0000","last_clean":"2024-03-16T08:16:33.629218+0000","last_became_active":"2024-03-16T08:15:02.166621+0000","last_became_peered":"2024-03-16T08:15:02.166621+0000","last_unstale":"2024-03-16T08:16:33.629218+0000","last_undegraded":"2024-03-16T08:16:33.629218+0000","last_fullsized":"2024-03-16T08:16:33.629218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:28.018824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388595+0000","last_change":"2024-03-16T08:15:01.167404+0000","last_active":"2024-03-16T08:16:05.388595+0000","last_peered":"2024-03-16T08:16:05.388595+0000","last_clean":"2024-03-16T08:16:05.388595+0000","last_became_active":"2024-03-16T08:15:01.166020+0000","last_became_peered":"2024-03-16T08:15:01.166020+0000","last_unstale":"2024-03-16T08:16:05.388595+0000","last_undegraded":"2024-03-16T08:16:05.388595+0000","last_fullsized":"2024-03-16T08:16:05.388595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:45.492295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628965+0000","last_change":"2024-03-16T08:15:02.169386+0000","last_active":"2024-03-16T08:16:33.628965+0000","last_peered":"2024-03-16T08:16:33.628965+0000","last_clean":"2024-03-16T08:16:33.628965+0000","last_became_active":"2024-03-16T08:15:02.166945+0000","last_became_peered":"2024-03-16T08:15:02.166945+0000","last_unstale":"2024-03-16T08:16:33.628965+0000","last_undegraded":"2024-03-16T08:16:33.628965+0000","last_fullsized":"2024-03-16T08:16:33.628965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:55:21.036209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:59.716883+0000","last_change":"2024-03-16T08:15:01.163476+0000","last_active":"2024-03-16T08:16:59.716883+0000","last_peered":"2024-03-16T08:16:59.716883+0000","last_clean":"2024-03-16T08:16:59.716883+0000","last_became_active":"2024-03-16T08:15:01.163155+0000","last_became_peered":"2024-03-16T08:15:01.163155+0000","last_unstale":"2024-03-16T08:16:59.716883+0000","last_undegraded":"2024-03-16T08:16:59.716883+0000","last_fullsized":"2024-03-16T08:16:59.716883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:22:02.304298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.615961+0000","last_change":"2024-03-16T08:15:02.174229+0000","last_active":"2024-03-16T08:16:33.615961+0000","last_peered":"2024-03-16T08:16:33.615961+0000","last_clean":"2024-03-16T08:16:33.615961+0000","last_became_active":"2024-03-16T08:15:02.173910+0000","last_became_peered":"2024-03-16T08:15:02.173910+0000","last_unstale":"2024-03-16T08:16:33.615961+0000","last_undegraded":"2024-03-16T08:16:33.615961+0000","last_fullsized":"2024-03-16T08:16:33.615961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:12.591280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387994+0000","last_change":"2024-03-16T08:15:01.169722+0000","last_active":"2024-03-16T08:16:05.387994+0000","last_peered":"2024-03-16T08:16:05.387994+0000","last_clean":"2024-03-16T08:16:05.387994+0000","last_became_active":"2024-03-16T08:15:01.169434+0000","last_became_peered":"2024-03-16T08:15:01.169434+0000","last_unstale":"2024-03-16T08:16:05.387994+0000","last_undegraded":"2024-03-16T08:16:05.387994+0000","last_fullsized":"2024-03-16T08:16:05.387994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:56:57.465214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636796+0000","last_change":"2024-03-16T08:15:02.174189+0000","last_active":"2024-03-16T08:16:33.636796+0000","last_peered":"2024-03-16T08:16:33.636796+0000","last_clean":"2024-03-16T08:16:33.636796+0000","last_became_active":"2024-03-16T08:15:02.171876+0000","last_became_peered":"2024-03-16T08:15:02.171876+0000","last_unstale":"2024-03-16T08:16:33.636796+0000","last_undegraded":"2024-03-16T08:16:33.636796+0000","last_fullsized":"2024-03-16T08:16:33.636796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:26:08.972263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.394865+0000","last_change":"2024-03-16T08:16:05.394457+0000","last_active":"2024-03-16T08:16:05.394865+0000","last_peered":"2024-03-16T08:16:05.394865+0000","last_clean":"2024-03-16T08:16:05.394865+0000","last_became_active":"2024-03-16T08:16:05.390239+0000","last_became_peered":"2024-03-16T08:16:05.390239+0000","last_unstale":"2024-03-16T08:16:05.394865+0000","last_undegraded":"2024-03-16T08:16:05.394865+0000","last_fullsized":"2024-03-16T08:16:05.394865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:29:29.442138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631505+0000","last_change":"2024-03-16T08:15:02.171816+0000","last_active":"2024-03-16T08:16:33.631505+0000","last_peered":"2024-03-16T08:16:33.631505+0000","last_clean":"2024-03-16T08:16:33.631505+0000","last_became_active":"2024-03-16T08:15:02.167494+0000","last_became_peered":"2024-03-16T08:15:02.167494+0000","last_unstale":"2024-03-16T08:16:33.631505+0000","last_undegraded":"2024-03-16T08:16:33.631505+0000","last_fullsized":"2024-03-16T08:16:33.631505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:52.811969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.390744+0000","last_change":"2024-03-16T08:16:05.390744+0000","last_active":"2024-03-16T08:16:05.390744+0000","last_peered":"2024-03-16T08:16:05.390744+0000","last_clean":"2024-03-16T08:16:05.390744+0000","last_became_active":"2024-03-16T08:16:05.390378+0000","last_became_peered":"2024-03-16T08:16:05.390378+0000","last_unstale":"2024-03-16T08:16:05.390744+0000","last_undegraded":"2024-03-16T08:16:05.390744+0000","last_fullsized":"2024-03-16T08:16:05.390744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:13:29.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.617438+0000","last_change":"2024-03-16T08:15:02.173218+0000","last_active":"2024-03-16T08:16:33.617438+0000","last_peered":"2024-03-16T08:16:33.617438+0000","last_clean":"2024-03-16T08:16:33.617438+0000","last_became_active":"2024-03-16T08:15:02.172872+0000","last_became_peered":"2024-03-16T08:15:02.172872+0000","last_unstale":"2024-03-16T08:16:33.617438+0000","last_undegraded":"2024-03-16T08:16:33.617438+0000","last_fullsized":"2024-03-16T08:16:33.617438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:59.638675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380895+0000","last_change":"2024-03-16T08:15:01.159609+0000","last_active":"2024-03-16T08:16:05.380895+0000","last_peered":"2024-03-16T08:16:05.380895+0000","last_clean":"2024-03-16T08:16:05.380895+0000","last_became_active":"2024-03-16T08:15:01.159206+0000","last_became_peered":"2024-03-16T08:15:01.159206+0000","last_unstale":"2024-03-16T08:16:05.380895+0000","last_undegraded":"2024-03-16T08:16:05.380895+0000","last_fullsized":"2024-03-16T08:16:05.380895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:00.785043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.620223+0000","last_change":"2024-03-16T08:16:05.389020+0000","last_active":"2024-03-16T08:16:33.620223+0000","last_peered":"2024-03-16T08:16:33.620223+0000","last_clean":"2024-03-16T08:16:33.620223+0000","last_became_active":"2024-03-16T08:16:05.388668+0000","last_became_peered":"2024-03-16T08:16:05.388668+0000","last_unstale":"2024-03-16T08:16:33.620223+0000","last_undegraded":"2024-03-16T08:16:33.620223+0000","last_fullsized":"2024-03-16T08:16:33.620223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:34:04.661388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633962+0000","last_change":"2024-03-16T08:15:02.167734+0000","last_active":"2024-03-16T08:16:33.633962+0000","last_peered":"2024-03-16T08:16:33.633962+0000","last_clean":"2024-03-16T08:16:33.633962+0000","last_became_active":"2024-03-16T08:15:02.167408+0000","last_became_peered":"2024-03-16T08:15:02.167408+0000","last_unstale":"2024-03-16T08:16:33.633962+0000","last_undegraded":"2024-03-16T08:16:33.633962+0000","last_fullsized":"2024-03-16T08:16:33.633962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:57.197400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":136,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:58.715510+0000","last_change":"2024-03-16T08:15:01.171573+0000","last_active":"2024-03-16T08:16:58.715510+0000","last_peered":"2024-03-16T08:16:58.715510+0000","last_clean":"2024-03-16T08:16:58.715510+0000","last_became_active":"2024-03-16T08:15:01.171245+0000","last_became_peered":"2024-03-16T08:15:01.171245+0000","last_unstale":"2024-03-16T08:16:58.715510+0000","last_undegraded":"2024-03-16T08:16:58.715510+0000","last_fullsized":"2024-03-16T08:16:58.715510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:31.128709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.636330+0000","last_change":"2024-03-16T08:15:02.179927+0000","last_active":"2024-03-16T08:16:33.636330+0000","last_peered":"2024-03-16T08:16:33.636330+0000","last_clean":"2024-03-16T08:16:33.636330+0000","last_became_active":"2024-03-16T08:15:02.179586+0000","last_became_peered":"2024-03-16T08:15:02.179586+0000","last_unstale":"2024-03-16T08:16:33.636330+0000","last_undegraded":"2024-03-16T08:16:33.636330+0000","last_fullsized":"2024-03-16T08:16:33.636330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:43:45.580687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387927+0000","last_change":"2024-03-16T08:15:01.168427+0000","last_active":"2024-03-16T08:16:05.387927+0000","last_peered":"2024-03-16T08:16:05.387927+0000","last_clean":"2024-03-16T08:16:05.387927+0000","last_became_active":"2024-03-16T08:15:01.168066+0000","last_became_peered":"2024-03-16T08:15:01.168066+0000","last_unstale":"2024-03-16T08:16:05.387927+0000","last_undegraded":"2024-03-16T08:16:05.387927+0000","last_fullsized":"2024-03-16T08:16:05.387927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:31:30.361079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380865+0000","last_change":"2024-03-16T08:13:49.887572+0000","last_active":"2024-03-16T08:16:05.380865+0000","last_peered":"2024-03-16T08:16:05.380865+0000","last_clean":"2024-03-16T08:16:05.380865+0000","last_became_active":"2024-03-16T08:13:49.887164+0000","last_became_peered":"2024-03-16T08:13:49.887164+0000","last_unstale":"2024-03-16T08:16:05.380865+0000","last_undegraded":"2024-03-16T08:16:05.380865+0000","last_fullsized":"2024-03-16T08:16:05.380865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:10:28.361178+0000","last_clean_scrub_stamp":"2024-03-16T08:10:28.361178+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:07.839434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.627112+0000","last_change":"2024-03-16T08:15:02.157046+0000","last_active":"2024-03-16T08:16:33.627112+0000","last_peered":"2024-03-16T08:16:33.627112+0000","last_clean":"2024-03-16T08:16:33.627112+0000","last_became_active":"2024-03-16T08:15:02.156677+0000","last_became_peered":"2024-03-16T08:15:02.156677+0000","last_unstale":"2024-03-16T08:16:33.627112+0000","last_undegraded":"2024-03-16T08:16:33.627112+0000","last_fullsized":"2024-03-16T08:16:33.627112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:12:25.286957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380918+0000","last_change":"2024-03-16T08:15:01.170777+0000","last_active":"2024-03-16T08:16:05.380918+0000","last_peered":"2024-03-16T08:16:05.380918+0000","last_clean":"2024-03-16T08:16:05.380918+0000","last_became_active":"2024-03-16T08:15:01.170443+0000","last_became_peered":"2024-03-16T08:15:01.170443+0000","last_unstale":"2024-03-16T08:16:05.380918+0000","last_undegraded":"2024-03-16T08:16:05.380918+0000","last_fullsized":"2024-03-16T08:16:05.380918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:00.048618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388079+0000","last_change":"2024-03-16T08:15:01.170285+0000","last_active":"2024-03-16T08:16:05.388079+0000","last_peered":"2024-03-16T08:16:05.388079+0000","last_clean":"2024-03-16T08:16:05.388079+0000","last_became_active":"2024-03-16T08:15:01.169521+0000","last_became_peered":"2024-03-16T08:15:01.169521+0000","last_unstale":"2024-03-16T08:16:05.388079+0000","last_undegraded":"2024-03-16T08:16:05.388079+0000","last_fullsized":"2024-03-16T08:16:05.388079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:42.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616704+0000","last_change":"2024-03-16T08:15:02.165816+0000","last_active":"2024-03-16T08:16:33.616704+0000","last_peered":"2024-03-16T08:16:33.616704+0000","last_clean":"2024-03-16T08:16:33.616704+0000","last_became_active":"2024-03-16T08:15:02.164863+0000","last_became_peered":"2024-03-16T08:15:02.164863+0000","last_unstale":"2024-03-16T08:16:33.616704+0000","last_undegraded":"2024-03-16T08:16:33.616704+0000","last_fullsized":"2024-03-16T08:16:33.616704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:41.082260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.380845+0000","last_change":"2024-03-16T08:15:01.163191+0000","last_active":"2024-03-16T08:16:05.380845+0000","last_peered":"2024-03-16T08:16:05.380845+0000","last_clean":"2024-03-16T08:16:05.380845+0000","last_became_active":"2024-03-16T08:15:01.162845+0000","last_became_peered":"2024-03-16T08:15:01.162845+0000","last_unstale":"2024-03-16T08:16:05.380845+0000","last_undegraded":"2024-03-16T08:16:05.380845+0000","last_fullsized":"2024-03-16T08:16:05.380845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:51:43.403910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624878+0000","last_change":"2024-03-16T08:15:02.174725+0000","last_active":"2024-03-16T08:16:33.624878+0000","last_peered":"2024-03-16T08:16:33.624878+0000","last_clean":"2024-03-16T08:16:33.624878+0000","last_became_active":"2024-03-16T08:15:02.171086+0000","last_became_peered":"2024-03-16T08:15:02.171086+0000","last_unstale":"2024-03-16T08:16:33.624878+0000","last_undegraded":"2024-03-16T08:16:33.624878+0000","last_fullsized":"2024-03-16T08:16:33.624878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:31.058947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388054+0000","last_change":"2024-03-16T08:15:01.171537+0000","last_active":"2024-03-16T08:16:05.388054+0000","last_peered":"2024-03-16T08:16:05.388054+0000","last_clean":"2024-03-16T08:16:05.388054+0000","last_became_active":"2024-03-16T08:15:01.171212+0000","last_became_peered":"2024-03-16T08:15:01.171212+0000","last_unstale":"2024-03-16T08:16:05.388054+0000","last_undegraded":"2024-03-16T08:16:05.388054+0000","last_fullsized":"2024-03-16T08:16:05.388054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:12:57.788802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632266+0000","last_change":"2024-03-16T08:15:02.180836+0000","last_active":"2024-03-16T08:16:33.632266+0000","last_peered":"2024-03-16T08:16:33.632266+0000","last_clean":"2024-03-16T08:16:33.632266+0000","last_became_active":"2024-03-16T08:15:02.180517+0000","last_became_peered":"2024-03-16T08:15:02.180517+0000","last_unstale":"2024-03-16T08:16:33.632266+0000","last_undegraded":"2024-03-16T08:16:33.632266+0000","last_fullsized":"2024-03-16T08:16:33.632266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:17:14.805782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.652991+0000","last_change":"2024-03-16T08:15:01.165394+0000","last_active":"2024-03-16T08:16:05.652991+0000","last_peered":"2024-03-16T08:16:05.652991+0000","last_clean":"2024-03-16T08:16:05.652991+0000","last_became_active":"2024-03-16T08:15:01.165069+0000","last_became_peered":"2024-03-16T08:15:01.165069+0000","last_unstale":"2024-03-16T08:16:05.652991+0000","last_undegraded":"2024-03-16T08:16:05.652991+0000","last_fullsized":"2024-03-16T08:16:05.652991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:41.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.585204+0000","last_change":"2024-03-16T08:15:02.172713+0000","last_active":"2024-03-16T08:16:33.585204+0000","last_peered":"2024-03-16T08:16:33.585204+0000","last_clean":"2024-03-16T08:16:33.585204+0000","last_became_active":"2024-03-16T08:15:02.172384+0000","last_became_peered":"2024-03-16T08:15:02.172384+0000","last_unstale":"2024-03-16T08:16:33.585204+0000","last_undegraded":"2024-03-16T08:16:33.585204+0000","last_fullsized":"2024-03-16T08:16:33.585204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384150+0000","last_change":"2024-03-16T08:15:01.173437+0000","last_active":"2024-03-16T08:16:05.384150+0000","last_peered":"2024-03-16T08:16:05.384150+0000","last_clean":"2024-03-16T08:16:05.384150+0000","last_became_active":"2024-03-16T08:15:01.172876+0000","last_became_peered":"2024-03-16T08:15:01.172876+0000","last_unstale":"2024-03-16T08:16:05.384150+0000","last_undegraded":"2024-03-16T08:16:05.384150+0000","last_fullsized":"2024-03-16T08:16:05.384150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:06.502655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.625869+0000","last_change":"2024-03-16T08:15:02.174878+0000","last_active":"2024-03-16T08:16:33.625869+0000","last_peered":"2024-03-16T08:16:33.625869+0000","last_clean":"2024-03-16T08:16:33.625869+0000","last_became_active":"2024-03-16T08:15:02.172992+0000","last_became_peered":"2024-03-16T08:15:02.172992+0000","last_unstale":"2024-03-16T08:16:33.625869+0000","last_undegraded":"2024-03-16T08:16:33.625869+0000","last_fullsized":"2024-03-16T08:16:33.625869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:36:13.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653334+0000","last_change":"2024-03-16T08:15:01.171142+0000","last_active":"2024-03-16T08:16:05.653334+0000","last_peered":"2024-03-16T08:16:05.653334+0000","last_clean":"2024-03-16T08:16:05.653334+0000","last_became_active":"2024-03-16T08:15:01.170628+0000","last_became_peered":"2024-03-16T08:15:01.170628+0000","last_unstale":"2024-03-16T08:16:05.653334+0000","last_undegraded":"2024-03-16T08:16:05.653334+0000","last_fullsized":"2024-03-16T08:16:05.653334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:18:44.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.628250+0000","last_change":"2024-03-16T08:15:02.175992+0000","last_active":"2024-03-16T08:16:33.628250+0000","last_peered":"2024-03-16T08:16:33.628250+0000","last_clean":"2024-03-16T08:16:33.628250+0000","last_became_active":"2024-03-16T08:15:02.175623+0000","last_became_peered":"2024-03-16T08:15:02.175623+0000","last_unstale":"2024-03-16T08:16:33.628250+0000","last_undegraded":"2024-03-16T08:16:33.628250+0000","last_fullsized":"2024-03-16T08:16:33.628250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:13.248472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.387802+0000","last_change":"2024-03-16T08:15:01.170072+0000","last_active":"2024-03-16T08:16:05.387802+0000","last_peered":"2024-03-16T08:16:05.387802+0000","last_clean":"2024-03-16T08:16:05.387802+0000","last_became_active":"2024-03-16T08:15:01.169814+0000","last_became_peered":"2024-03-16T08:15:01.169814+0000","last_unstale":"2024-03-16T08:16:05.387802+0000","last_undegraded":"2024-03-16T08:16:05.387802+0000","last_fullsized":"2024-03-16T08:16:05.387802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:03:58.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.637947+0000","last_change":"2024-03-16T08:15:02.172636+0000","last_active":"2024-03-16T08:16:33.637947+0000","last_peered":"2024-03-16T08:16:33.637947+0000","last_clean":"2024-03-16T08:16:33.637947+0000","last_became_active":"2024-03-16T08:15:02.172269+0000","last_became_peered":"2024-03-16T08:15:02.172269+0000","last_unstale":"2024-03-16T08:16:33.637947+0000","last_undegraded":"2024-03-16T08:16:33.637947+0000","last_fullsized":"2024-03-16T08:16:33.637947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:50.375387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.653465+0000","last_change":"2024-03-16T08:15:01.170035+0000","last_active":"2024-03-16T08:16:05.653465+0000","last_peered":"2024-03-16T08:16:05.653465+0000","last_clean":"2024-03-16T08:16:05.653465+0000","last_became_active":"2024-03-16T08:15:01.169735+0000","last_became_peered":"2024-03-16T08:15:01.169735+0000","last_unstale":"2024-03-16T08:16:05.653465+0000","last_undegraded":"2024-03-16T08:16:05.653465+0000","last_fullsized":"2024-03-16T08:16:05.653465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:25.881746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.638034+0000","last_change":"2024-03-16T08:15:02.171453+0000","last_active":"2024-03-16T08:16:33.638034+0000","last_peered":"2024-03-16T08:16:33.638034+0000","last_clean":"2024-03-16T08:16:33.638034+0000","last_became_active":"2024-03-16T08:15:02.171120+0000","last_became_peered":"2024-03-16T08:15:02.171120+0000","last_unstale":"2024-03-16T08:16:33.638034+0000","last_undegraded":"2024-03-16T08:16:33.638034+0000","last_fullsized":"2024-03-16T08:16:33.638034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:31.159549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.388862+0000","last_change":"2024-03-16T08:15:01.166138+0000","last_active":"2024-03-16T08:16:05.388862+0000","last_peered":"2024-03-16T08:16:05.388862+0000","last_clean":"2024-03-16T08:16:05.388862+0000","last_became_active":"2024-03-16T08:15:01.163304+0000","last_became_peered":"2024-03-16T08:15:01.163304+0000","last_unstale":"2024-03-16T08:16:05.388862+0000","last_undegraded":"2024-03-16T08:16:05.388862+0000","last_fullsized":"2024-03-16T08:16:05.388862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:30.752810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.619455+0000","last_change":"2024-03-16T08:15:02.173824+0000","last_active":"2024-03-16T08:16:33.619455+0000","last_peered":"2024-03-16T08:16:33.619455+0000","last_clean":"2024-03-16T08:16:33.619455+0000","last_became_active":"2024-03-16T08:15:02.173475+0000","last_became_peered":"2024-03-16T08:15:02.173475+0000","last_unstale":"2024-03-16T08:16:33.619455+0000","last_undegraded":"2024-03-16T08:16:33.619455+0000","last_fullsized":"2024-03-16T08:16:33.619455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:52.787141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:05.384729+0000","last_change":"2024-03-16T08:15:01.165163+0000","last_active":"2024-03-16T08:16:05.384729+0000","last_peered":"2024-03-16T08:16:05.384729+0000","last_clean":"2024-03-16T08:16:05.384729+0000","last_became_active":"2024-03-16T08:15:01.164780+0000","last_became_peered":"2024-03-16T08:15:01.164780+0000","last_unstale":"2024-03-16T08:16:05.384729+0000","last_undegraded":"2024-03-16T08:16:05.384729+0000","last_fullsized":"2024-03-16T08:16:05.384729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:00.142606+0000","last_clean_scrub_stamp":"2024-03-16T08:15:00.142606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:25.945309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.630956+0000","last_change":"2024-03-16T08:15:02.168955+0000","last_active":"2024-03-16T08:16:33.630956+0000","last_peered":"2024-03-16T08:16:33.630956+0000","last_clean":"2024-03-16T08:16:33.630956+0000","last_became_active":"2024-03-16T08:15:02.168641+0000","last_became_peered":"2024-03-16T08:15:02.168641+0000","last_unstale":"2024-03-16T08:16:33.630956+0000","last_undegraded":"2024-03-16T08:16:33.630956+0000","last_fullsized":"2024-03-16T08:16:33.630956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:17:38.422629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624759+0000","last_change":"2024-03-16T08:15:02.175519+0000","last_active":"2024-03-16T08:16:33.624759+0000","last_peered":"2024-03-16T08:16:33.624759+0000","last_clean":"2024-03-16T08:16:33.624759+0000","last_became_active":"2024-03-16T08:15:02.174990+0000","last_became_peered":"2024-03-16T08:15:02.174990+0000","last_unstale":"2024-03-16T08:16:33.624759+0000","last_undegraded":"2024-03-16T08:16:33.624759+0000","last_fullsized":"2024-03-16T08:16:33.624759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:02.960266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.631321+0000","last_change":"2024-03-16T08:15:02.171272+0000","last_active":"2024-03-16T08:16:33.631321+0000","last_peered":"2024-03-16T08:16:33.631321+0000","last_clean":"2024-03-16T08:16:33.631321+0000","last_became_active":"2024-03-16T08:15:02.170343+0000","last_became_peered":"2024-03-16T08:15:02.170343+0000","last_unstale":"2024-03-16T08:16:33.631321+0000","last_undegraded":"2024-03-16T08:16:33.631321+0000","last_fullsized":"2024-03-16T08:16:33.631321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:44:38.608543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.622948+0000","last_change":"2024-03-16T08:15:02.168921+0000","last_active":"2024-03-16T08:16:33.622948+0000","last_peered":"2024-03-16T08:16:33.622948+0000","last_clean":"2024-03-16T08:16:33.622948+0000","last_became_active":"2024-03-16T08:15:02.168538+0000","last_became_peered":"2024-03-16T08:15:02.168538+0000","last_unstale":"2024-03-16T08:16:33.622948+0000","last_undegraded":"2024-03-16T08:16:33.622948+0000","last_fullsized":"2024-03-16T08:16:33.622948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:57:03.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633266+0000","last_change":"2024-03-16T08:15:02.176084+0000","last_active":"2024-03-16T08:16:33.633266+0000","last_peered":"2024-03-16T08:16:33.633266+0000","last_clean":"2024-03-16T08:16:33.633266+0000","last_became_active":"2024-03-16T08:15:02.175179+0000","last_became_peered":"2024-03-16T08:15:02.175179+0000","last_unstale":"2024-03-16T08:16:33.633266+0000","last_undegraded":"2024-03-16T08:16:33.633266+0000","last_fullsized":"2024-03-16T08:16:33.633266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:48:13.000100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.632885+0000","last_change":"2024-03-16T08:15:02.173676+0000","last_active":"2024-03-16T08:16:33.632885+0000","last_peered":"2024-03-16T08:16:33.632885+0000","last_clean":"2024-03-16T08:16:33.632885+0000","last_became_active":"2024-03-16T08:15:02.173296+0000","last_became_peered":"2024-03-16T08:15:02.173296+0000","last_unstale":"2024-03-16T08:16:33.632885+0000","last_undegraded":"2024-03-16T08:16:33.632885+0000","last_fullsized":"2024-03-16T08:16:33.632885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:40:58.146160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.623433+0000","last_change":"2024-03-16T08:15:02.172829+0000","last_active":"2024-03-16T08:16:33.623433+0000","last_peered":"2024-03-16T08:16:33.623433+0000","last_clean":"2024-03-16T08:16:33.623433+0000","last_became_active":"2024-03-16T08:15:02.172461+0000","last_became_peered":"2024-03-16T08:15:02.172461+0000","last_unstale":"2024-03-16T08:16:33.623433+0000","last_undegraded":"2024-03-16T08:16:33.623433+0000","last_fullsized":"2024-03-16T08:16:33.623433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:03.317497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639557+0000","last_change":"2024-03-16T08:15:02.163549+0000","last_active":"2024-03-16T08:16:33.639557+0000","last_peered":"2024-03-16T08:16:33.639557+0000","last_clean":"2024-03-16T08:16:33.639557+0000","last_became_active":"2024-03-16T08:15:02.163234+0000","last_became_peered":"2024-03-16T08:15:02.163234+0000","last_unstale":"2024-03-16T08:16:33.639557+0000","last_undegraded":"2024-03-16T08:16:33.639557+0000","last_fullsized":"2024-03-16T08:16:33.639557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:30.855440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.616343+0000","last_change":"2024-03-16T08:16:05.654782+0000","last_active":"2024-03-16T08:16:33.616343+0000","last_peered":"2024-03-16T08:16:33.616343+0000","last_clean":"2024-03-16T08:16:33.616343+0000","last_became_active":"2024-03-16T08:16:05.654117+0000","last_became_peered":"2024-03-16T08:16:05.654117+0000","last_unstale":"2024-03-16T08:16:33.616343+0000","last_undegraded":"2024-03-16T08:16:33.616343+0000","last_fullsized":"2024-03-16T08:16:33.616343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T08:17:45.678541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.614995+0000","last_change":"2024-03-16T08:15:02.171066+0000","last_active":"2024-03-16T08:16:33.614995+0000","last_peered":"2024-03-16T08:16:33.614995+0000","last_clean":"2024-03-16T08:16:33.614995+0000","last_became_active":"2024-03-16T08:15:02.168282+0000","last_became_peered":"2024-03-16T08:15:02.168282+0000","last_unstale":"2024-03-16T08:16:33.614995+0000","last_undegraded":"2024-03-16T08:16:33.614995+0000","last_fullsized":"2024-03-16T08:16:33.614995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:07:30.026573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.624368+0000","last_change":"2024-03-16T08:15:02.171224+0000","last_active":"2024-03-16T08:16:33.624368+0000","last_peered":"2024-03-16T08:16:33.624368+0000","last_clean":"2024-03-16T08:16:33.624368+0000","last_became_active":"2024-03-16T08:15:02.170869+0000","last_became_peered":"2024-03-16T08:15:02.170869+0000","last_unstale":"2024-03-16T08:16:33.624368+0000","last_undegraded":"2024-03-16T08:16:33.624368+0000","last_fullsized":"2024-03-16T08:16:33.624368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:55:59.396846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.639654+0000","last_change":"2024-03-16T08:15:02.168948+0000","last_active":"2024-03-16T08:16:33.639654+0000","last_peered":"2024-03-16T08:16:33.639654+0000","last_clean":"2024-03-16T08:16:33.639654+0000","last_became_active":"2024-03-16T08:15:02.167699+0000","last_became_peered":"2024-03-16T08:15:02.167699+0000","last_unstale":"2024-03-16T08:16:33.639654+0000","last_undegraded":"2024-03-16T08:16:33.639654+0000","last_fullsized":"2024-03-16T08:16:33.639654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:59.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.629430+0000","last_change":"2024-03-16T08:15:02.172762+0000","last_active":"2024-03-16T08:16:33.629430+0000","last_peered":"2024-03-16T08:16:33.629430+0000","last_clean":"2024-03-16T08:16:33.629430+0000","last_became_active":"2024-03-16T08:15:02.172417+0000","last_became_peered":"2024-03-16T08:15:02.172417+0000","last_unstale":"2024-03-16T08:16:33.629430+0000","last_undegraded":"2024-03-16T08:16:33.629430+0000","last_fullsized":"2024-03-16T08:16:33.629430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:12:13.322773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-16T08:16:33.633659+0000","last_change":"2024-03-16T08:15:02.167252+0000","last_active":"2024-03-16T08:16:33.633659+0000","last_peered":"2024-03-16T08:16:33.633659+0000","last_clean":"2024-03-16T08:16:33.633659+0000","last_became_active":"2024-03-16T08:15:02.166817+0000","last_became_peered":"2024-03-16T08:15:02.166817+0000","last_unstale":"2024-03-16T08:16:33.633659+0000","last_undegraded":"2024-03-16T08:16:33.633659+0000","last_fullsized":"2024-03-16T08:16:33.633659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:15:01.146523+0000","last_clean_scrub_stamp":"2024-03-16T08:15:01.146523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:40:25.296417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284156,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65437,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":416,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196619,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":459215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1111969,"data_compressed":37963,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051420,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":520214,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521788,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1051104,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34497,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36133,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61603,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63177,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62873,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:17:02.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- 172.21.15.49:0/990548077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 msgr2=0x7ff26c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 --2- 172.21.15.49:0/990548077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c109c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff264009ca0 tx=0x7ff26402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- 172.21.15.49:0/990548077 shutdown_connections 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 --2- 172.21.15.49:0/990548077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c109c50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 --2- 172.21.15.49:0/990548077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 --2- 172.21.15.49:0/990548077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- 172.21.15.49:0/990548077 >> 172.21.15.49:0/990548077 conn(0x7ff26c0fcdc0 msgr2=0x7ff26c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- 172.21.15.49:0/990548077 shutdown_connections 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- 172.21.15.49:0/990548077 wait complete. 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 Processor -- start 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.574+0000 7ff2722cc640 1 -- start start 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c10d470 con 0x7ff26c1033c0 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c10d2f0 con 0x7ff26c101810 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c10d5f0 con 0x7ff26c102a00 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58202/0 (socket says 172.21.15.49:58202) 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff26bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 -- 172.21.15.49:0/628742565 learned_addr learned my addr 172.21.15.49:0/628742565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48448/0 (socket says 172.21.15.49:48448) 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 -- 172.21.15.49:0/628742565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 msgr2=0x7ff26c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 -- 172.21.15.49:0/628742565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 msgr2=0x7ff26c075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 -- 172.21.15.49:0/628742565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c0749a0 con 0x7ff26c1033c0 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff26bfff640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff270869640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff264037040 tx=0x7ff264032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264041020 con 0x7ff26c1033c0 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff264034cd0 con 0x7ff26c1033c0 2024-03-16T08:17:02.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c1a1b40 con 0x7ff26c1033c0 2024-03-16T08:17:02.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.575+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264039040 con 0x7ff26c1033c0 2024-03-16T08:17:02.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.576+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a20e0 con 0x7ff26c1033c0 2024-03-16T08:17:02.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.576+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c108b30 con 0x7ff26c1033c0 2024-03-16T08:17:02.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.578+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff264044020 con 0x7ff26c1033c0 2024-03-16T08:17:02.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.578+0000 7ff2697fa640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.578+0000 7ff26bfff640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.578+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7ff2640f0850 con 0x7ff26c1033c0 2024-03-16T08:17:02.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.579+0000 7ff26bfff640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 0x7ff2580a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff26c102860 tx=0x7ff254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.583+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff26404b020 con 0x7ff26c1033c0 2024-03-16T08:17:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.767+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff26c0623e0 con 0x7ff26c1033c0 2024-03-16T08:17:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.769+0000 7ff2697fa640 1 -- 172.21.15.49:0/628742565 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7ff2640b7e20 con 0x7ff26c1033c0 2024-03-16T08:17:02.769 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:02.769 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 msgr2=0x7ff2580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 0x7ff2580a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff26c102860 tx=0x7ff254005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 msgr2=0x7ff26c074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff264037040 tx=0x7ff264032f40 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 shutdown_connections 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff26c1033c0 0x7ff26c074110 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c102a00 0x7ff26c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 --2- 172.21.15.49:0/628742565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff26c101810 0x7ff26c075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 >> 172.21.15.49:0/628742565 conn(0x7ff26c0fcdc0 msgr2=0x7ff26c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 shutdown_connections 2024-03-16T08:17:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.771+0000 7ff2722cc640 1 -- 172.21.15.49:0/628742565 wait complete. 2024-03-16T08:17:02.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:17:02.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 -- 172.21.15.49:0/2199374844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1029f0 msgr2=0x7fe3ac102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2199374844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac102e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009900 tx=0x7fe3a002bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 -- 172.21.15.49:0/2199374844 shutdown_connections 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2199374844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2199374844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac102e70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2199374844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac101800 0x7fe3ac101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 -- 172.21.15.49:0/2199374844 >> 172.21.15.49:0/2199374844 conn(0x7fe3ac0fcd70 msgr2=0x7fe3ac0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.894+0000 7fe3b2414640 1 -- 172.21.15.49:0/2199374844 shutdown_connections 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 -- 172.21.15.49:0/2199374844 wait complete. 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 Processor -- start 2024-03-16T08:17:02.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 -- start start 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac10d480 con 0x7fe3ac101800 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac10d300 con 0x7fe3ac1033b0 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3b2414640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac10d600 con 0x7fe3ac1029f0 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.895+0000 7fe3abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58226/0 (socket says 172.21.15.49:58226) 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 -- 172.21.15.49:0/2350719373 learned_addr learned my addr 172.21.15.49:0/2350719373 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3b09b1640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3ab7fe640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 -- 172.21.15.49:0/2350719373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac1029f0 msgr2=0x7fe3ac198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 -- 172.21.15.49:0/2350719373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 msgr2=0x7fe3ac19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 -- 172.21.15.49:0/2350719373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac19f750 con 0x7fe3ac101800 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3b09b1640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3abfff640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00e9c0 tx=0x7fe39c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00cd80 con 0x7fe3ac101800 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c0035f0 con 0x7fe3ac101800 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac19fa40 con 0x7fe3ac101800 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c010690 con 0x7fe3ac101800 2024-03-16T08:17:02.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.896+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac19ff10 con 0x7fe3ac101800 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.897+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac108b20 con 0x7fe3ac101800 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.898+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe39c0037a0 con 0x7fe3ac101800 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.899+0000 7fe3a97fa640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 0x7fe3940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.899+0000 7fe3ab7fe640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 0x7fe3940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.899+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe39c0ced90 con 0x7fe3ac101800 2024-03-16T08:17:02.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.900+0000 7fe3ab7fe640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 0x7fe3940a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002c7b0 tx=0x7fe3a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:02.903+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe39c09be60 con 0x7fe3ac101800 2024-03-16T08:17:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.086+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3ac062480 con 0x7fe3ac101800 2024-03-16T08:17:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.087+0000 7fe3a97fa640 1 -- 172.21.15.49:0/2350719373 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe39c09b760 con 0x7fe3ac101800 2024-03-16T08:17:03.087 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:17:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 msgr2=0x7fe3940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 0x7fe3940a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002c7b0 tx=0x7fe3a0005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 msgr2=0x7fe3ac197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00e9c0 tx=0x7fe39c00ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 shutdown_connections 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3ac1033b0 0x7fe3ac19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3ac1029f0 0x7fe3ac198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe3940a2800 0x7fe3940a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 --2- 172.21.15.49:0/2350719373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3ac101800 0x7fe3ac197b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 >> 172.21.15.49:0/2350719373 conn(0x7fe3ac0fcd70 msgr2=0x7fe3ac105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 shutdown_connections 2024-03-16T08:17:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.089+0000 7fe3b2414640 1 -- 172.21.15.49:0/2350719373 wait complete. 2024-03-16T08:17:03.101 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:17:03.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 -- 172.21.15.49:0/123866388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4104c50 msgr2=0x7fe5e4107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/123866388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4104c50 0x7fe5e4107040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 -- 172.21.15.49:0/123866388 shutdown_connections 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/123866388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 0x7fe5e4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/123866388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4104c50 0x7fe5e4107040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/123866388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4102320 0x7fe5e4104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 -- 172.21.15.49:0/123866388 >> 172.21.15.49:0/123866388 conn(0x7fe5e40fbc90 msgr2=0x7fe5e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 -- 172.21.15.49:0/123866388 shutdown_connections 2024-03-16T08:17:03.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.212+0000 7fe5e8c27640 1 -- 172.21.15.49:0/123866388 wait complete. 2024-03-16T08:17:03.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 Processor -- start 2024-03-16T08:17:03.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 -- start start 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4104c50 0x7fe5e4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 0x7fe5e419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d470 con 0x7fe5e4102320 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d2f0 con 0x7fe5e4107580 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e8c27640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d5f0 con 0x7fe5e4104c50 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58242/0 (socket says 172.21.15.49:58242) 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 -- 172.21.15.49:0/3082499083 learned_addr learned my addr 172.21.15.49:0/3082499083 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5dbfff640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4104c50 0x7fe5e4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e2d9d640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 0x7fe5e419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 -- 172.21.15.49:0/3082499083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4104c50 msgr2=0x7fe5e4198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4104c50 0x7fe5e4198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 -- 172.21.15.49:0/3082499083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 msgr2=0x7fe5e419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 0x7fe5e419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.213+0000 7fe5e259c640 1 -- 172.21.15.49:0/3082499083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e419f760 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5e259c640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400e9f0 tx=0x7fe5d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400cd80 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d4005500 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e419fa50 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4010690 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.214+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e419ff70 con 0x7fe5e4102320 2024-03-16T08:17:03.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.215+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e40fd780 con 0x7fe5e4102320 2024-03-16T08:17:03.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.216+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe5d40046c0 con 0x7fe5e4102320 2024-03-16T08:17:03.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.217+0000 7fe5db7fe640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 0x7fe5d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.217+0000 7fe5dbfff640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 0x7fe5d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.217+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fe5d40ced90 con 0x7fe5e4102320 2024-03-16T08:17:03.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.217+0000 7fe5dbfff640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 0x7fe5d00a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.221+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe5d409be60 con 0x7fe5e4102320 2024-03-16T08:17:03.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.403+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5e4062410 con 0x7fe5e4102320 2024-03-16T08:17:03.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.404+0000 7fe5db7fe640 1 -- 172.21.15.49:0/3082499083 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe5d409b760 con 0x7fe5e4102320 2024-03-16T08:17:03.404 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:17:03.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 msgr2=0x7fe5d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 0x7fe5d00a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 msgr2=0x7fe5e4197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400e9f0 tx=0x7fe5d400edf0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 shutdown_connections 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e4107580 0x7fe5e419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4104c50 0x7fe5e4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fe5d00a2880 0x7fe5d00a4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 --2- 172.21.15.49:0/3082499083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4102320 0x7fe5e4197b40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 >> 172.21.15.49:0/3082499083 conn(0x7fe5e40fbc90 msgr2=0x7fe5e4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 shutdown_connections 2024-03-16T08:17:03.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.406+0000 7fe5e8c27640 1 -- 172.21.15.49:0/3082499083 wait complete. 2024-03-16T08:17:03.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 -- 172.21.15.49:0/922609118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 msgr2=0x7f10fc100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 --2- 172.21.15.49:0/922609118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc100b20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009900 tx=0x7f10ec02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 -- 172.21.15.49:0/922609118 shutdown_connections 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 --2- 172.21.15.49:0/922609118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1022b0 0x7f10fc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 --2- 172.21.15.49:0/922609118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1018f0 0x7f10fc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 --2- 172.21.15.49:0/922609118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc100b20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 -- 172.21.15.49:0/922609118 >> 172.21.15.49:0/922609118 conn(0x7f10fc0fbc90 msgr2=0x7f10fc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 -- 172.21.15.49:0/922609118 shutdown_connections 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.529+0000 7f1100d48640 1 -- 172.21.15.49:0/922609118 wait complete. 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 Processor -- start 2024-03-16T08:17:03.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 -- start start 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1018f0 0x7f10fc102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 0x7f10fc103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc10c3d0 con 0x7f10fc100700 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc10c250 con 0x7f10fc1022b0 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f1100d48640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc10c550 con 0x7f10fc1018f0 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58262/0 (socket says 172.21.15.49:58262) 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 -- 172.21.15.49:0/1179820486 learned_addr learned my addr 172.21.15.49:0/1179820486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10f3fff640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1018f0 0x7f10fc102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fad9d640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 0x7f10fc103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 -- 172.21.15.49:0/1179820486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1018f0 msgr2=0x7f10fc102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1018f0 0x7f10fc102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 -- 172.21.15.49:0/1179820486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 msgr2=0x7f10fc103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 0x7f10fc103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 -- 172.21.15.49:0/1179820486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc103aa0 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fad9d640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 0x7f10fc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.530+0000 7f10fa59c640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.531+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec043020 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.531+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ec0054e0 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.531+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc1a1c50 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.531+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003c40 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.531+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1a21a0 con 0x7f10fc100700 2024-03-16T08:17:03.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.532+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc1a2370 con 0x7f10fc100700 2024-03-16T08:17:03.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.533+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f10ec046020 con 0x7f10fc100700 2024-03-16T08:17:03.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.533+0000 7f10f37fe640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 0x7f10e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.534+0000 7f10f3fff640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 0x7f10e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.534+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f10ec0f0a40 con 0x7f10fc100700 2024-03-16T08:17:03.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.534+0000 7f10f3fff640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 0x7f10e80a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10fc104400 tx=0x7f10e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.538+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f10ec04d020 con 0x7f10fc100700 2024-03-16T08:17:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.724+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10fc1a2540 con 0x7f10fc100700 2024-03-16T08:17:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.724+0000 7f10f37fe640 1 -- 172.21.15.49:0/1179820486 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f10ec0bdb10 con 0x7f10fc100700 2024-03-16T08:17:03.724 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/628742565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2350719373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:03.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3082499083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 msgr2=0x7f10e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 0x7f10e80a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10fc104400 tx=0x7f10e4005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 msgr2=0x7f10fc104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec02c650 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 shutdown_connections 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10fc1022b0 0x7f10fc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10fc1018f0 0x7f10fc102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f10e80a2880 0x7f10e80a4d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 --2- 172.21.15.49:0/1179820486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10fc100700 0x7f10fc104690 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 >> 172.21.15.49:0/1179820486 conn(0x7f10fc0fbc90 msgr2=0x7f10fc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 shutdown_connections 2024-03-16T08:17:03.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.726+0000 7f1100d48640 1 -- 172.21.15.49:0/1179820486 wait complete. 2024-03-16T08:17:03.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:17:03.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/628742565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:03.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2350719373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:03.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3082499083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:03.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:17:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 -- 172.21.15.49:0/3359363687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 msgr2=0x7fa9141099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 --2- 172.21.15.49:0/3359363687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa9141099b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009ca0 tx=0x7fa90c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 -- 172.21.15.49:0/3359363687 shutdown_connections 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 --2- 172.21.15.49:0/3359363687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa9141099b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 --2- 172.21.15.49:0/3359363687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914104c30 0x7fa914107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 --2- 172.21.15.49:0/3359363687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914102300 0x7fa9141046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 -- 172.21.15.49:0/3359363687 >> 172.21.15.49:0/3359363687 conn(0x7fa9140fbd10 msgr2=0x7fa9140fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 -- 172.21.15.49:0/3359363687 shutdown_connections 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.846+0000 7fa918edc640 1 -- 172.21.15.49:0/3359363687 wait complete. 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 Processor -- start 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 -- start start 2024-03-16T08:17:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 0x7fa914197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 0x7fa914197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410d490 con 0x7fa914107560 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410d310 con 0x7fa914102300 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa918edc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410d610 con 0x7fa914104c30 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58278/0 (socket says 172.21.15.49:58278) 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 -- 172.21.15.49:0/3906517120 learned_addr learned my addr 172.21.15.49:0/3906517120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa91259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 0x7fa914197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa911d9b640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 0x7fa914197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 -- 172.21.15.49:0/3906517120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 msgr2=0x7fa914197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 0x7fa914197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 -- 172.21.15.49:0/3906517120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 msgr2=0x7fa914197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 0x7fa914197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa912d9d640 1 -- 172.21.15.49:0/3906517120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419f6b0 con 0x7fa914107560 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa911d9b640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 0x7fa914197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.847+0000 7fa91259c640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 0x7fa914197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa912d9d640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa90c037040 tx=0x7fa90c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c041020 con 0x7fa914107560 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419f940 con 0x7fa914107560 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c034cd0 con 0x7fa914107560 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c039040 con 0x7fa914107560 2024-03-16T08:17:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.848+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91419fe30 con 0x7fa914107560 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.849+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9140fd800 con 0x7fa914107560 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.850+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa90c044020 con 0x7fa914107560 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.851+0000 7fa90b7fe640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 0x7fa9000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.851+0000 7fa91259c640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 0x7fa9000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.851+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fa90c0f0920 con 0x7fa914107560 2024-03-16T08:17:03.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.851+0000 7fa91259c640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 0x7fa9000a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009fc0 tx=0x7fa8fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:03.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:03.856+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa90c04b020 con 0x7fa914107560 2024-03-16T08:17:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T08:17:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:17:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/628742565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2350719373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:17:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3082499083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:17:04.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.088+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa91402d390 con 0x7fa914107560 2024-03-16T08:17:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.089+0000 7fa90b7fe640 1 -- 172.21.15.49:0/3906517120 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fa90c0bda80 con 0x7fa914107560 2024-03-16T08:17:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:04.089 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:17:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:17:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 msgr2=0x7fa9000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 0x7fa9000a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009fc0 tx=0x7fa8fc005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 msgr2=0x7fa91419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa90c037040 tx=0x7fa90c032f40 comp rx=0 tx=0).stop 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 shutdown_connections 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fa9000a2800 0x7fa9000a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa914107560 0x7fa91419f040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa914104c30 0x7fa914197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 --2- 172.21.15.49:0/3906517120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa914102300 0x7fa914197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 >> 172.21.15.49:0/3906517120 conn(0x7fa9140fbd10 msgr2=0x7fa914102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 shutdown_connections 2024-03-16T08:17:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.091+0000 7fa918edc640 1 -- 172.21.15.49:0/3906517120 wait complete. 2024-03-16T08:17:04.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.210+0000 7fad7a5c4640 1 -- 172.21.15.49:0/2861651972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74101800 msgr2=0x7fad74101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.210+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/2861651972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74101800 0x7fad74101c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad64009860 tx=0x7fad6402bb70 comp rx=0 tx=0).stop 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- 172.21.15.49:0/2861651972 shutdown_connections 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/2861651972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad741033b0 0x7fad74109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/2861651972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 0x7fad74102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/2861651972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74101800 0x7fad74101c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- 172.21.15.49:0/2861651972 >> 172.21.15.49:0/2861651972 conn(0x7fad740fcd90 msgr2=0x7fad740ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- 172.21.15.49:0/2861651972 shutdown_connections 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- 172.21.15.49:0/2861651972 wait complete. 2024-03-16T08:17:04.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 Processor -- start 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- start start 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 0x7fad74198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad741033b0 0x7fad7419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad7410d450 con 0x7fad74101800 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad7410d2d0 con 0x7fad741029f0 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.211+0000 7fad7a5c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad7410d5d0 con 0x7fad741033b0 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58292/0 (socket says 172.21.15.49:58292) 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 -- 172.21.15.49:0/576104647 learned_addr learned my addr 172.21.15.49:0/576104647 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad737fe640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 0x7fad74198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad78b61640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad741033b0 0x7fad7419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 -- 172.21.15.49:0/576104647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad741033b0 msgr2=0x7fad7419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad741033b0 0x7fad7419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 -- 172.21.15.49:0/576104647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 msgr2=0x7fad74198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 0x7fad74198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 -- 172.21.15.49:0/576104647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7419f770 con 0x7fad74101800 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad73fff640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad6402c0b0 tx=0x7fad64032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64041020 con 0x7fad74101800 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6402c9f0 con 0x7fad74101800 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7419fa00 con 0x7fad74101800 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64039040 con 0x7fad74101800 2024-03-16T08:17:04.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.212+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7419fef0 con 0x7fad74101800 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.213+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad74108bc0 con 0x7fad74101800 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.214+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fad64044020 con 0x7fad74101800 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.215+0000 7fad717fa640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 0x7fad5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.215+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fad640f06f0 con 0x7fad74101800 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.215+0000 7fad737fe640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 0x7fad5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.216+0000 7fad737fe640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 0x7fad5c0a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fad74199060 tx=0x7fad68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.219+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fad6404b020 con 0x7fad74101800 2024-03-16T08:17:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.405+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad740623e0 con 0x7fad74101800 2024-03-16T08:17:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.407+0000 7fad717fa640 1 -- 172.21.15.49:0/576104647 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22588 (secure 0 0 0) 0x7fad640bd7c0 con 0x7fad74101800 2024-03-16T08:17:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:04.407 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:16:05.377762+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:17:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 msgr2=0x7fad5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 0x7fad5c0a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fad74199060 tx=0x7fad68005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 msgr2=0x7fad74197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad6402c0b0 tx=0x7fad64032430 comp rx=0 tx=0).stop 2024-03-16T08:17:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 shutdown_connections 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad741033b0 0x7fad7419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad741029f0 0x7fad74198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fad5c0a2800 0x7fad5c0a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 --2- 172.21.15.49:0/576104647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad74101800 0x7fad74197b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 >> 172.21.15.49:0/576104647 conn(0x7fad740fcd90 msgr2=0x7fad74105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 shutdown_connections 2024-03-16T08:17:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.409+0000 7fad7a5c4640 1 -- 172.21.15.49:0/576104647 wait complete. 2024-03-16T08:17:04.421 DEBUG:tasks.fwd_scrub:Yielding 2024-03-16T08:17:04.421 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T08:17:04.432 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T08:17:04.432 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:17:04.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-16T08:17:04.433 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-16T08:17:04.433 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:17:04.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:17:04.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:17:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 -- 172.21.15.49:0/1964003807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70100700 msgr2=0x7f1c70100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 --2- 172.21.15.49:0/1964003807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70100700 0x7f1c70100b20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009900 tx=0x7f1c6c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 -- 172.21.15.49:0/1964003807 shutdown_connections 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 --2- 172.21.15.49:0/1964003807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701022b0 0x7f1c70108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 --2- 172.21.15.49:0/1964003807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c701018f0 0x7f1c70101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 --2- 172.21.15.49:0/1964003807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c70100700 0x7f1c70100b20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 -- 172.21.15.49:0/1964003807 >> 172.21.15.49:0/1964003807 conn(0x7f1c700fbc70 msgr2=0x7f1c700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 -- 172.21.15.49:0/1964003807 shutdown_connections 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.542+0000 7f1c7733b640 1 -- 172.21.15.49:0/1964003807 wait complete. 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 Processor -- start 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 -- start start 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 0x7f1c70197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 0x7f1c7019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010c350 con 0x7f1c701022b0 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010c1d0 con 0x7f1c701018f0 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c7733b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010c4d0 con 0x7f1c70100700 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c748d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c758d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 0x7f1c7019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c748d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:34494/0 (socket says 172.21.15.49:34494) 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c750d7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 0x7f1c70197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.543+0000 7f1c748d6640 1 -- 172.21.15.49:0/244149325 learned_addr learned my addr 172.21.15.49:0/244149325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 -- 172.21.15.49:0/244149325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 msgr2=0x7f1c70197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 -- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 msgr2=0x7f1c7019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 0x7f1c7019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 -- 172.21.15.49:0/244149325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7019f6a0 con 0x7f1c70100700 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c748d6640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c758d8640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 0x7f1c7019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c750d7640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 0x7f1c70197a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c02c170 tx=0x7f1c6c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c041070 con 0x7f1c70100700 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6c004450 con 0x7f1c70100700 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7019f930 con 0x7f1c70100700 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003e50 con 0x7f1c70100700 2024-03-16T08:17:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.544+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7019fdf0 con 0x7f1c70100700 2024-03-16T08:17:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.545+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7019fff0 con 0x7f1c70100700 2024-03-16T08:17:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 -- 172.21.15.49:0/284230718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f8103a20 msgr2=0x7f27f8103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 --2- 172.21.15.49:0/284230718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f8103a20 0x7f27f8103e40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009860 tx=0x7f27f402bb70 comp rx=0 tx=0).stop 2024-03-16T08:17:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 -- 172.21.15.49:0/284230718 shutdown_connections 2024-03-16T08:17:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 --2- 172.21.15.49:0/284230718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f81055d0 0x7f27f810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 --2- 172.21.15.49:0/284230718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8104c10 0x7f27f8105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 --2- 172.21.15.49:0/284230718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f8103a20 0x7f27f8103e40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 -- 172.21.15.49:0/284230718 >> 172.21.15.49:0/284230718 conn(0x7f27f80fef90 msgr2=0x7f27f81013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 -- 172.21.15.49:0/284230718 shutdown_connections 2024-03-16T08:17:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 -- 172.21.15.49:0/284230718 wait complete. 2024-03-16T08:17:04.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1c6c044070 con 0x7f1c70100700 2024-03-16T08:17:04.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.546+0000 7f280029a640 1 Processor -- start 2024-03-16T08:17:04.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 -- start start 2024-03-16T08:17:04.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 0x7f27f819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81055d0 0x7f27f81a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810f670 con 0x7f27f8103a20 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810f4f0 con 0x7f27f8104c10 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f280029a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810f7f0 con 0x7f27f81055d0 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58324/0 (socket says 172.21.15.49:58324) 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 -- 172.21.15.49:0/190836030 learned_addr learned my addr 172.21.15.49:0/190836030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fd835640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 0x7f27f819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe837640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81055d0 0x7f27f81a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f1c667fc640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 0x7f1c5c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f1c748d6640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 0x7f1c5c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 -- 172.21.15.49:0/190836030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81055d0 msgr2=0x7f27f81a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81055d0 0x7f27f81a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 -- 172.21.15.49:0/190836030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 msgr2=0x7f27f819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 0x7f27f819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 -- 172.21.15.49:0/190836030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f81a1960 con 0x7f27f8103a20 2024-03-16T08:17:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fd835640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 0x7f27f819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f1c6c0f0530 con 0x7f1c70100700 2024-03-16T08:17:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.547+0000 7f27fe036640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f27f402c0b0 tx=0x7f27f4032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f1c748d6640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 0x7f1c5c0a4f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c600095f0 tx=0x7f1c60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f27ef7fe640 1 -- 172.21.15.49:0/190836030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4041020 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f27ef7fe640 1 -- 172.21.15.49:0/190836030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f402c9f0 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a1bf0 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f27ef7fe640 1 -- 172.21.15.49:0/190836030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4039040 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.548+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a2130 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.550+0000 7f27ef7fe640 1 -- 172.21.15.49:0/190836030 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f27f4044020 con 0x7f27f8103a20 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.550+0000 7f2802f63740 1 Processor -- start 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 -- start start 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c65042980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c64f9f0d0 con 0x563c6503b380 2024-03-16T08:17:04.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c64f47c70 con 0x563c650404f0 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f2802f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c64f9b170 con 0x563c6503a990 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27ef7fe640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 0x7f27e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe837640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe837640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48600/0 (socket says 172.21.15.49:48600) 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58332/0 (socket says 172.21.15.49:58332) 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe837640 1 -- 172.21.15.49:0/2452972905 learned_addr learned my addr 172.21.15.49:0/2452972905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fd835640 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c65042980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fe837640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 0x7f27e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27ef7fe640 1 -- 172.21.15.49:0/190836030 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f27f40f0300 con 0x7f27f8103a20 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fd835640 1 -- 172.21.15.49:0/2452972905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 msgr2=0x563c6503adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f27fd835640 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fd835640 1 -- 172.21.15.49:0/2452972905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 msgr2=0x563c6503fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fd835640 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.551+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1c6c0bcf50 con 0x7f1c70100700 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fd835640 1 -- 172.21.15.49:0/2452972905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c64d7b7e0 con 0x563c650404f0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fe837640 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fd835640 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c65042980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009530 tx=0x7f27e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4de86906732421c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27fe837640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 0x7f27e40a4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x563c6503bbe0 tx=0x7f27f0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27ecbfe640 1 -- 172.21.15.49:0/2452972905 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8004220 con 0x563c650404f0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27ecbfe640 1 -- 172.21.15.49:0/2452972905 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e80043c0 con 0x563c650404f0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.552+0000 7f27ecbfe640 1 -- 172.21.15.49:0/2452972905 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e800fb40 con 0x563c650404f0 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- 172.21.15.49:0/2452972905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 msgr2=0x563c65042980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c65042980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009530 tx=0x7f27e8011810 comp rx=0 tx=0).stop 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- 172.21.15.49:0/2452972905 shutdown_connections 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c65042980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 --2- 172.21.15.49:0/2452972905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- 172.21.15.49:0/2452972905 >> 172.21.15.49:0/2452972905 conn(0x563c65002c50 msgr2=0x563c65036300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- 172.21.15.49:0/2452972905 shutdown_connections 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- 172.21.15.49:0/2452972905 wait complete. 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 -- start start 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.553+0000 7f2802f63740 1 Processor -- start 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c65044ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c6503b380 0x563c65045410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c650434a0 con 0x563c650404f0 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c64f5b0d0 con 0x563c6503b380 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c65043620 con 0x563c6503a990 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f2802f63740 10 client.0 _ll_register_callbacks cb 0x563c64f24760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe036640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c65044ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58338/0 (socket says 172.21.15.49:58338) 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 -- 172.21.15.49:0/3569458019 learned_addr learned my addr 172.21.15.49:0/3569458019 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fd835640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c6503b380 0x563c65045410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 -- 172.21.15.49:0/3569458019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 msgr2=0x563c65044ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c65044ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 -- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c6503b380 msgr2=0x563c65045410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c6503b380 0x563c65045410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 -- 172.21.15.49:0/3569458019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c64ddbf70 con 0x563c650404f0 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe036640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c65044ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27fe837640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27f0004260 tx=0x7f27f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc269a94c4460e52 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00109b0 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.554+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f27f0010b50 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e80043c0 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f27f0010d50 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 10 client.15390 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c64c73170 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e80043c0 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f27f0027520 con 0x563c650404f0 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 20 client.15390 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c65052e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 4 client.15390 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c64fac610 con 0x563c650509b0 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.555+0000 7f27fe036640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c65052e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.556+0000 7f27e1ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.556+0000 7f27fe036640 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c65052e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0998b53deb1a8be server_cookie=6895343014d75965 in_seq=0 out_seq=0 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f27e1ffb640 1 -- 172.21.15.49:0/3569458019 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563c64fac610 con 0x563c650509b0 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f27e1ffb640 10 client.15390 handle_command_reply: tid=0 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 client.15390 shutdown 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 msgr2=0x563c6504c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27f0004260 tx=0x7f27f0004a40 comp rx=0 tx=0).stop 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 shutdown_connections 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c65052e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f27e1ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c650404f0 0x563c6504c3e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c6503b380 0x563c65045410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 --2- 172.21.15.49:0/3569458019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c65044ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 >> 172.21.15.49:0/3569458019 conn(0x563c65002c50 msgr2=0x563c650354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 shutdown_connections 2024-03-16T08:17:04.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 1 -- 172.21.15.49:0/3569458019 wait complete. 2024-03-16T08:17:04.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.557+0000 7f2802f63740 20 client.15390 trim_cache size 0 max 16384 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 Processor -- start 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 -- start start 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a9330 con 0x563c6503b380 2024-03-16T08:17:04.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a6c00 con 0x563c650404f0 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f2802f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a4a50 con 0x563c6503a990 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe036640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe837640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe036640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48610/0 (socket says 172.21.15.49:48610) 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe036640 1 -- 172.21.15.49:0/373791442 learned_addr learned my addr 172.21.15.49:0/373791442 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 -- 172.21.15.49:0/373791442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 msgr2=0x563c6503c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 -- 172.21.15.49:0/373791442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 msgr2=0x563c6504c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fd835640 1 -- 172.21.15.49:0/373791442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c650368a0 con 0x563c6503b380 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe837640 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.575+0000 7f27fe036640 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.576+0000 7f27fd835640 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503ce00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006010 tx=0x7f27e8009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fbc65093465025 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.576+0000 7f27e3fff640 1 -- 172.21.15.49:0/373791442 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8029070 con 0x563c6503b380 2024-03-16T08:17:04.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.576+0000 7f27e3fff640 1 -- 172.21.15.49:0/373791442 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e8005040 con 0x563c6503b380 2024-03-16T08:17:04.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.576+0000 7f27e3fff640 1 -- 172.21.15.49:0/373791442 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003b30 con 0x563c6503b380 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- 172.21.15.49:0/373791442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 msgr2=0x563c6503ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503ce00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006010 tx=0x7f27e8009c40 comp rx=0 tx=0).stop 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- 172.21.15.49:0/373791442 shutdown_connections 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503b380 0x563c6503ce00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- 172.21.15.49:0/373791442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503a990 0x563c6503c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- 172.21.15.49:0/373791442 >> 172.21.15.49:0/373791442 conn(0x563c65002c50 msgr2=0x563c65044ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- 172.21.15.49:0/373791442 shutdown_connections 2024-03-16T08:17:04.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- 172.21.15.49:0/373791442 wait complete. 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- start start 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 Processor -- start 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503b380 0x563c65049cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a50c0 con 0x563c6503a990 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a6530 con 0x563c650404f0 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81a5790 con 0x563c6503b380 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f2802f63740 10 client.0 _ll_register_callbacks cb 0x7f27f000f520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f27fe036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58352/0 (socket says 172.21.15.49:58352) 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f27fe036640 1 -- 172.21.15.49:0/404004961 learned_addr learned my addr 172.21.15.49:0/404004961 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f27fe837640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.577+0000 7f27fd835640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503b380 0x563c65049cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 -- 172.21.15.49:0/404004961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503b380 msgr2=0x563c65049cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503b380 0x563c65049cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 -- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 msgr2=0x563c6504ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 -- 172.21.15.49:0/404004961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6504b270 con 0x563c6503a990 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe837640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27fe036640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009830 tx=0x7f27f40045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188d2f41d7fec9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f40d0020 con 0x563c6503a990 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f27f40042c0 con 0x563c6503a990 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e8005040 con 0x563c6503a990 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f27f40c0dc0 con 0x563c6503a990 2024-03-16T08:17:04.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f2802f63740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c6504c620 con 0x563c6503a990 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.578+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e8005040 con 0x563c6503a990 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f27f403c030 con 0x563c6503a990 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 20 client.15396 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c650b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 4 client.15396 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c65031b40 con 0x563c650509b0 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f27fd835640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c650b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f27fd835640 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c650b93c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed045a93bddb702 server_cookie=b4aaa67a0c8753c6 in_seq=0 out_seq=0 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.579+0000 7f27e1ffb640 10 client.15396 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f27e1ffb640 1 -- 172.21.15.49:0/404004961 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c65031b40 con 0x563c650509b0 2024-03-16T08:17:04.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f27e1ffb640 10 client.15396 handle_command_reply: tid=0 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 client.15396 shutdown 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 msgr2=0x563c65049780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009830 tx=0x7f27f40045c0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 shutdown_connections 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c650509b0 0x563c650b93c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f27e1ffb640 0 client.15396 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c650404f0 0x563c6504ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c6503b380 0x563c65049cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 --2- 172.21.15.49:0/404004961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c6503a990 0x563c65049780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 >> 172.21.15.49:0/404004961 conn(0x563c65002c50 msgr2=0x563c65044e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 shutdown_connections 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 1 -- 172.21.15.49:0/404004961 wait complete. 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.580+0000 7f2802f63740 20 client.15396 trim_cache size 0 max 16384 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "6b176dc9-14bb-4c67-b577-768c749dde6a", 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:17:04.587 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 msgr2=0x7f27e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 0x7f27e40a4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x563c6503bbe0 tx=0x7f27f0004ab0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 msgr2=0x7f27f8199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f27f402c0b0 tx=0x7f27f4032430 comp rx=0 tx=0).stop 2024-03-16T08:17:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 shutdown_connections 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81055d0 0x7f27f81a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27f8104c10 0x7f27f819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f27e40a2880 0x7f27e40a4d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 --2- 172.21.15.49:0/190836030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f8103a20 0x7f27f8199d40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 >> 172.21.15.49:0/190836030 conn(0x7f27f80fef90 msgr2=0x7f27f8107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 shutdown_connections 2024-03-16T08:17:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.581+0000 7f280029a640 1 -- 172.21.15.49:0/190836030 wait complete. 2024-03-16T08:17:04.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:17:04.592 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 -- 172.21.15.49:0/1858803713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c107560 msgr2=0x7fab9c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 --2- 172.21.15.49:0/1858803713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c107560 0x7fab9c109950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fab94009be0 tx=0x7fab9402bef0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 -- 172.21.15.49:0/1858803713 shutdown_connections 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 --2- 172.21.15.49:0/1858803713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c107560 0x7fab9c109950 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 --2- 172.21.15.49:0/1858803713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c104c30 0x7fab9c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 --2- 172.21.15.49:0/1858803713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 -- 172.21.15.49:0/1858803713 >> 172.21.15.49:0/1858803713 conn(0x7fab9c0fbc70 msgr2=0x7fab9c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 -- 172.21.15.49:0/1858803713 shutdown_connections 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.703+0000 7faba1aff640 1 -- 172.21.15.49:0/1858803713 wait complete. 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 Processor -- start 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 -- start start 2024-03-16T08:17:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c107560 0x7fab9c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10d480 con 0x7fab9c104c30 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10d300 con 0x7fab9c102300 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7faba1aff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10d600 con 0x7fab9c107560 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7fab9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7fab9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58370/0 (socket says 172.21.15.49:58370) 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.704+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 -- 172.21.15.49:0/2511378656 learned_addr learned my addr 172.21.15.49:0/2511378656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c107560 0x7fab9c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 -- 172.21.15.49:0/2511378656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c107560 msgr2=0x7fab9c1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c107560 0x7fab9c1a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 -- 172.21.15.49:0/2511378656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 msgr2=0x7fab9c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 -- 172.21.15.49:0/2511378656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c1a1920 con 0x7fab9c104c30 2024-03-16T08:17:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab9affd640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab9000e9c0 tx=0x7fab9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab98ff9640 1 -- 172.21.15.49:0/2511378656 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9000ccc0 con 0x7fab9c104c30 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab98ff9640 1 -- 172.21.15.49:0/2511378656 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab90005500 con 0x7fab9c104c30 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7fab98ff9640 1 -- 172.21.15.49:0/2511378656 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90010690 con 0x7fab9c104c30 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c1a1c10 con 0x7fab9c104c30 2024-03-16T08:17:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.705+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c1a2110 con 0x7fab9c104c30 2024-03-16T08:17:04.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 Processor -- start 2024-03-16T08:17:04.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7fab98ff9640 1 -- 172.21.15.49:0/2511378656 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fab900046c0 con 0x7fab9c104c30 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 -- start start 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564194936500 con 0x5641949d6cd0 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641948debe0 con 0x5641949d1b60 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.708+0000 7faba47c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641949325a0 con 0x5641949d1170 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58382/0 (socket says 172.21.15.49:58382) 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 -- 172.21.15.49:0/1591780179 learned_addr learned my addr 172.21.15.49:0/1591780179 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 -- 172.21.15.49:0/1591780179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 msgr2=0x5641949d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab98ff9640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 0x7fab840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 -- 172.21.15.49:0/1591780179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 msgr2=0x5641949d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 -- 172.21.15.49:0/1591780179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641948f24b0 con 0x5641949d6cd0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 0x7fab840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab98ff9640 1 -- 172.21.15.49:0/2511378656 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fab900ced30 con 0x7fab9c104c30 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab9affd640 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fab9009de40 tx=0x7fab900979d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cac42396fe3c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab81ffb640 1 -- 172.21.15.49:0/1591780179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9009b4d0 con 0x5641949d6cd0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab81ffb640 1 -- 172.21.15.49:0/1591780179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9009ca30 con 0x5641949d6cd0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.709+0000 7fab81ffb640 1 -- 172.21.15.49:0/1591780179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9009df10 con 0x5641949d6cd0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- 172.21.15.49:0/1591780179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 msgr2=0x5641949d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 0x7fab840a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab94004ab0 tx=0x7fab94004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fab9009de40 tx=0x7fab900979d0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- 172.21.15.49:0/1591780179 shutdown_connections 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949d9160 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 --2- 172.21.15.49:0/1591780179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- 172.21.15.49:0/1591780179 >> 172.21.15.49:0/1591780179 conn(0x5641949cc6e0 msgr2=0x5641949ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- 172.21.15.49:0/1591780179 shutdown_connections 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- 172.21.15.49:0/1591780179 wait complete. 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.710+0000 7faba47c6740 1 -- start start 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 Processor -- start 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949db620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5641949d9bc0 con 0x5641949d6cd0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641948f2160 con 0x5641949d1b60 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641949d9d40 con 0x5641949d1170 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7faba47c6740 10 client.0 _ll_register_callbacks cb 0x5641948d6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58384/0 (socket says 172.21.15.49:58384) 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 -- 172.21.15.49:0/941646151 learned_addr learned my addr 172.21.15.49:0/941646151 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 -- 172.21.15.49:0/941646151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 msgr2=0x5641949db620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949db620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 -- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 msgr2=0x5641949dbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949dbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9bfff640 1 -- 172.21.15.49:0/941646151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641947147d0 con 0x5641949d6cd0 2024-03-16T08:17:04.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949db620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.711+0000 7fab9affd640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab9bfff640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab94035000 tx=0x7fab94009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f94248bafc5499 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94051070 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fab94033dc0 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564194943a40 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab9405d050 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7faba47c6740 10 client.15408 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56419464a3f0 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564194943a40 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.712+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fab940349c0 con 0x5641949d6cd0 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:04.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 20 client.15408 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5641949e70d0 0x5641949e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 4 client.15408 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419493dd90 con 0x5641949e70d0 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5641949e70d0 0x5641949e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7fab7affd640 10 client.15408 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.713+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5641949e70d0 0x5641949e9550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0f31bc6ed8846e server_cookie=9919059a8f1c16be in_seq=0 out_seq=0 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7fab7affd640 1 -- 172.21.15.49:0/941646151 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56419493dd90 con 0x5641949e70d0 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7fab7affd640 10 client.15408 handle_command_reply: tid=0 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 client.15408 shutdown 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 msgr2=0x5641949e2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab94035000 tx=0x7fab94009500 comp rx=0 tx=0).stop 2024-03-16T08:17:04.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 shutdown_connections 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5641949e70d0 0x5641949e9550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x5641949e2ba0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1b60 0x5641949dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7fab7affd640 0 client.15408 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 --2- 172.21.15.49:0/941646151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1170 0x5641949db620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 >> 172.21.15.49:0/941646151 conn(0x5641949cc6e0 msgr2=0x5641949cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 shutdown_connections 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.714+0000 7faba47c6740 1 -- 172.21.15.49:0/941646151 wait complete. 2024-03-16T08:17:04.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.715+0000 7faba47c6740 20 client.15408 trim_cache size 0 max 16384 2024-03-16T08:17:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1179820486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3906517120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/576104647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 Processor -- start 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 -- start start 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d6cd0 0x5641949cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ad760 con 0x5641949d1170 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ab030 con 0x5641949d6cd0 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.746+0000 7faba47c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1a8e80 con 0x5641949d1b60 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58394/0 (socket says 172.21.15.49:58394) 2024-03-16T08:17:04.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 -- 172.21.15.49:0/1592559708 learned_addr learned my addr 172.21.15.49:0/1592559708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9bfff640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d6cd0 0x5641949cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9affd640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 -- 172.21.15.49:0/1592559708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 msgr2=0x5641949e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 -- 172.21.15.49:0/1592559708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d6cd0 msgr2=0x5641949cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d6cd0 0x5641949cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 -- 172.21.15.49:0/1592559708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419460a0f0 con 0x5641949d1170 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9affd640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5641949d25d0 tx=0x7fab8c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e3fb65718cec2e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab817fa640 1 -- 172.21.15.49:0/1592559708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c028070 con 0x5641949d1170 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab817fa640 1 -- 172.21.15.49:0/1592559708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c022650 con 0x5641949d1170 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.747+0000 7fab817fa640 1 -- 172.21.15.49:0/1592559708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c010820 con 0x5641949d1170 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- 172.21.15.49:0/1592559708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 msgr2=0x5641949c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5641949d25d0 tx=0x7fab8c005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:04.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- 172.21.15.49:0/1592559708 shutdown_connections 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d6cd0 0x5641949cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 --2- 172.21.15.49:0/1592559708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d1170 0x5641949c78e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- 172.21.15.49:0/1592559708 >> 172.21.15.49:0/1592559708 conn(0x5641949cc6e0 msgr2=0x5641949d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- 172.21.15.49:0/1592559708 shutdown_connections 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- 172.21.15.49:0/1592559708 wait complete. 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 -- start start 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.748+0000 7faba47c6740 1 Processor -- start 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 0x5641949dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1a94f0 con 0x5641949d6cd0 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1aa960 con 0x5641949d1170 2024-03-16T08:17:04.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1a9bc0 con 0x5641949d1b60 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 10 client.0 _ll_register_callbacks cb 0x7fab94009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 0x5641949dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58406/0 (socket says 172.21.15.49:58406) 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 0x5641949dfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:34594/0 (socket says 172.21.15.49:34594) 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 -- 172.21.15.49:0/2075563334 learned_addr learned my addr 172.21.15.49:0/2075563334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 -- 172.21.15.49:0/2075563334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 msgr2=0x5641949e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 -- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 msgr2=0x5641949dfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 0x5641949dfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 -- 172.21.15.49:0/2075563334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641948fd9d0 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab9bfff640 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab9402c430 tx=0x7fab9404d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15edb57afabe082 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab7affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94065070 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7fab9404d940 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.749+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c022650 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab94071050 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564194a4cab0 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c022650 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fab94034430 con 0x5641949d6cd0 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 20 client.15414 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564194a4d430 0x564194a4f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:04.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 4 client.15414 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641948bb540 con 0x564194a4d430 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.750+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564194a4d430 0x564194a4f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7fab9b7fe640 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564194a4d430 0x564194a4f850 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d804e5faf3227aa server_cookie=cd9c5fa1f426ee39 in_seq=0 out_seq=0 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7fab7affd640 10 client.15414 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7fab7affd640 1 -- 172.21.15.49:0/2075563334 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5641948bb540 con 0x564194a4d430 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7fab7affd640 10 client.15414 handle_command_reply: tid=0 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 client.15414 shutdown 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 msgr2=0x564194a4b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab9402c430 tx=0x7fab9404d6d0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 shutdown_connections 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564194a4d430 0x564194a4f850 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641949d6cd0 0x564194a4b440 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7fab7affd640 0 client.15414 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:04.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641949d1b60 0x5641949e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 --2- 172.21.15.49:0/2075563334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641949d1170 0x5641949dfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 >> 172.21.15.49:0/2075563334 conn(0x5641949cc6e0 msgr2=0x5641949cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 shutdown_connections 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 1 -- 172.21.15.49:0/2075563334 wait complete. 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.751+0000 7faba47c6740 20 client.15414 trim_cache size 0 max 16384 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "6b176dc9-14bb-4c67-b577-768c749dde6a": { 2024-03-16T08:17:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:17:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "6b176dc9-14bb-4c67-b577-768c749dde6a", 2024-03-16T08:17:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:17:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:17:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:17:04.759 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 msgr2=0x7fab840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 0x7fab840a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab94004ab0 tx=0x7fab94004a40 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 msgr2=0x7fab9c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab9000e9c0 tx=0x7fab9000ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 shutdown_connections 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab9c107560 0x7fab9c1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7fab840a2800 0x7fab840a4cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab9c104c30 0x7fab9c19a250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 --2- 172.21.15.49:0/2511378656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab9c102300 0x7fab9c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 >> 172.21.15.49:0/2511378656 conn(0x7fab9c0fbc70 msgr2=0x7fab9c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 shutdown_connections 2024-03-16T08:17:04.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.752+0000 7faba1aff640 1 -- 172.21.15.49:0/2511378656 wait complete. 2024-03-16T08:17:04.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1179820486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:04.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3906517120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:04.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/576104647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:04.763 INFO:tasks.cephfs.filesystem:scrub status for tag:6b176dc9-14bb-4c67-b577-768c749dde6a - {'path': '/', 'tag': '6b176dc9-14bb-4c67-b577-768c749dde6a', 'options': 'recursive,force'} 2024-03-16T08:17:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:04.809+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1c701a01c0 con 0x7f1c70100700 2024-03-16T08:17:04.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1179820486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:17:04.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3906517120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:04.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/576104647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:17:05.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.464+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f1c6c0035f0 con 0x7f1c70100700 2024-03-16T08:17:05.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.466+0000 7f1c667fc640 1 -- 172.21.15.49:0/244149325 <== mon.1 v2:172.21.15.154:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f1c6c0ba5a0 con 0x7f1c70100700 2024-03-16T08:17:05.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 msgr2=0x7f1c5c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 0x7f1c5c0a4f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c600095f0 tx=0x7f1c60005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 msgr2=0x7f1c70197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 0x7f1c70197a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c02c170 tx=0x7f1c6c02c650 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 shutdown_connections 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f1c5c0a2a40 0x7f1c5c0a4f00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c701022b0 0x7f1c7019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c701018f0 0x7f1c70197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 --2- 172.21.15.49:0/244149325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70100700 0x7f1c70197a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 >> 172.21.15.49:0/244149325 conn(0x7f1c700fbc70 msgr2=0x7f1c700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 shutdown_connections 2024-03-16T08:17:05.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.469+0000 7f1c7733b640 1 -- 172.21.15.49:0/244149325 wait complete. 2024-03-16T08:17:05.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.611+0000 7f965e5f3640 1 -- 172.21.15.49:0/1422810896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 msgr2=0x7f9658101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.611+0000 7f965e5f3640 1 --2- 172.21.15.49:0/1422810896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f9658101850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ca0 tx=0x7f965002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 -- 172.21.15.49:0/1422810896 shutdown_connections 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 --2- 172.21.15.49:0/1422810896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f9658101850 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 --2- 172.21.15.49:0/1422810896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9658100a30 0x7f9658100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 --2- 172.21.15.49:0/1422810896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96581000d0 0x7f96581004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 -- 172.21.15.49:0/1422810896 >> 172.21.15.49:0/1422810896 conn(0x7f96580fbc70 msgr2=0x7f96580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 -- 172.21.15.49:0/1422810896 shutdown_connections 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 -- 172.21.15.49:0/1422810896 wait complete. 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 Processor -- start 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.612+0000 7f965e5f3640 1 -- start start 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 0x7f9658199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9658100a30 0x7f965819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f965810f670 con 0x7f96581013d0 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f965810f4f0 con 0x7f96581000d0 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f965810f7f0 con 0x7f9658100a30 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f9657fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 0x7f9658199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58436/0 (socket says 172.21.15.49:58436) 2024-03-16T08:17:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 -- 172.21.15.49:0/441193339 learned_addr learned my addr 172.21.15.49:0/441193339 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f96577fe640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9658100a30 0x7f965819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 -- 172.21.15.49:0/441193339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9658100a30 msgr2=0x7f965819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9658100a30 0x7f965819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 -- 172.21.15.49:0/441193339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 msgr2=0x7f9658199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 0x7f9658199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 -- 172.21.15.49:0/441193339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96581a1830 con 0x7f96581013d0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f9657fff640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 0x7f9658199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965cb90640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9650037040 tx=0x7f9650032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650041020 con 0x7f96581013d0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581a1ac0 con 0x7f96581013d0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.613+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9650034cd0 con 0x7f96581013d0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.614+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650039040 con 0x7f96581013d0 2024-03-16T08:17:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.614+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581a1fb0 con 0x7f96581013d0 2024-03-16T08:17:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.615+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965810ae20 con 0x7f96581013d0 2024-03-16T08:17:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.616+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f9650044020 con 0x7f96581013d0 2024-03-16T08:17:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.616+0000 7f96557fa640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 0x7f96400a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.616+0000 7f9657fff640 1 -- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 msgr2=0x7f96400a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.616+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f96500f09c0 con 0x7f96581013d0 2024-03-16T08:17:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.616+0000 7f9657fff640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 0x7f96400a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:17:05.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.620+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f96500f2e20 con 0x7f96581013d0 2024-03-16T08:17:05.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:17:05.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/244149325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mon[36471]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: ignoring --setuser ceph since I am not root 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: ignoring --setgroup ceph since I am not root 2024-03-16T08:17:05.726 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:05 smithi154 ceph-mgr[38368]: -- 172.21.15.154:0/928018892 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac9bb885a0 con 0x55ac9bb5cc00 2024-03-16T08:17:05.787 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setuser ceph since I am not root 2024-03-16T08:17:05.787 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: ignoring --setgroup ceph since I am not root 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/244149325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:05 smithi049 ceph-mon[31942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.808+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9658062480 con 0x7f96581013d0 2024-03-16T08:17:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.812+0000 7f96557fa640 1 -- 172.21.15.49:0/441193339 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f965004b020 con 0x7f96581013d0 2024-03-16T08:17:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 msgr2=0x7f96400a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:17:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 0x7f96400a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 msgr2=0x7f96581a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9650037040 tx=0x7f9650032f40 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 shutdown_connections 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f96400a28f0 0x7f96400a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96581013d0 0x7f96581a1190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9658100a30 0x7f965819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 --2- 172.21.15.49:0/441193339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96581000d0 0x7f9658199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 >> 172.21.15.49:0/441193339 conn(0x7f96580fbc70 msgr2=0x7f96580fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 shutdown_connections 2024-03-16T08:17:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.814+0000 7f965e5f3640 1 -- 172.21.15.49:0/441193339 wait complete. 2024-03-16T08:17:05.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-16T08:17:05.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:17:05.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/244149325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mon[36561]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: ignoring --setuser ceph since I am not root 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: ignoring --setgroup ceph since I am not root 2024-03-16T08:17:05.861 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:05 smithi129 ceph-mgr[37986]: -- 172.21.15.129:0/2482490082 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cffbc2c5a0 con 0x55cffbc01000 2024-03-16T08:17:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 -- 172.21.15.49:0/1495141524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 msgr2=0x7f06d8109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 --2- 172.21.15.49:0/1495141524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d8109930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009ca0 tx=0x7f06d002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 -- 172.21.15.49:0/1495141524 shutdown_connections 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 --2- 172.21.15.49:0/1495141524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d8109930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 --2- 172.21.15.49:0/1495141524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d8104bf0 0x7f06d8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 --2- 172.21.15.49:0/1495141524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d81022c0 0x7f06d81046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 -- 172.21.15.49:0/1495141524 >> 172.21.15.49:0/1495141524 conn(0x7f06d80fbc70 msgr2=0x7f06d80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 -- 172.21.15.49:0/1495141524 shutdown_connections 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.936+0000 7f06dcea1640 1 -- 172.21.15.49:0/1495141524 wait complete. 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 Processor -- start 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 -- start start 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 0x7f06d8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810d480 con 0x7f06d8107520 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810d300 con 0x7f06d81022c0 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06dcea1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810d600 con 0x7f06d8104bf0 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:34632/0 (socket says 172.21.15.49:34632) 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d659c640 1 -- 172.21.15.49:0/362377354 learned_addr learned my addr 172.21.15.49:0/362377354 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58450/0 (socket says 172.21.15.49:58450) 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.937+0000 7f06d5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 0x7f06d8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 msgr2=0x7f06d8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 0x7f06d8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 msgr2=0x7f06d819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 -- 172.21.15.49:0/362377354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d819f680 con 0x7f06d81022c0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d6d9d640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d5d9b640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 0x7f06d8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06d659c640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e9f0 tx=0x7f06c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c800cd80 con 0x7f06d81022c0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c8010910 con 0x7f06d81022c0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d819f970 con 0x7f06d81022c0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8018990 con 0x7f06d81022c0 2024-03-16T08:17:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.938+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d819fec0 con 0x7f06d81022c0 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.939+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d804f540 con 0x7f06d81022c0 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.940+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f06c8010ac0 con 0x7f06d81022c0 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.941+0000 7f06c77fe640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 0x7f06c00a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.941+0000 7f06d5d9b640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 msgr2=0x7f06c00a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.941+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f06c80cf1b0 con 0x7f06d81022c0 2024-03-16T08:17:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.941+0000 7f06d5d9b640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 0x7f06c00a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:17:05.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:05.944+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f06c809bc30 con 0x7f06d81022c0 2024-03-16T08:17:06.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.142+0000 7f06d5d9b640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 msgr2=0x7f06c00a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:06.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.142+0000 7f06d5d9b640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 0x7f06c00a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:17:06.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.148+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f06d80fd760 con 0x7f06d81022c0 2024-03-16T08:17:06.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.156+0000 7f06c77fe640 1 -- 172.21.15.49:0/362377354 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f06c809fcd0 con 0x7f06d81022c0 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 msgr2=0x7f06c00a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 0x7f06c00a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 msgr2=0x7f06d8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e9f0 tx=0x7f06c800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 shutdown_connections 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f06c00a28f0 0x7f06c00a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06d8107520 0x7f06d819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8104bf0 0x7f06d8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 --2- 172.21.15.49:0/362377354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06d81022c0 0x7f06d8197a60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 >> 172.21.15.49:0/362377354 conn(0x7f06d80fbc70 msgr2=0x7f06d8102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 shutdown_connections 2024-03-16T08:17:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.158+0000 7f06dcea1640 1 -- 172.21.15.49:0/362377354 wait complete. 2024-03-16T08:17:06.169 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3691920112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc1022b0 msgr2=0x7f21dc108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3691920112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc1022b0 0x7f21dc108b40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009ca0 tx=0x7f21d402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3691920112 shutdown_connections 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3691920112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc1022b0 0x7f21dc108b40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3691920112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1018f0 0x7f21dc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3691920112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21dc100700 0x7f21dc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3691920112 >> 172.21.15.49:0/3691920112 conn(0x7f21dc0fbc70 msgr2=0x7f21dc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3691920112 shutdown_connections 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.281+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3691920112 wait complete. 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 Processor -- start 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 -- start start 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 0x7f21dc104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21dc1018f0 0x7f21dc102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1022b0 0x7f21dc103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10c3a0 con 0x7f21dc100700 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10c220 con 0x7f21dc1018f0 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21e1f7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10c520 con 0x7f21dc1022b0 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 0x7f21dc104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 0x7f21dc104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58468/0 (socket says 172.21.15.49:58468) 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21db7fe640 1 -- 172.21.15.49:0/3128552758 learned_addr learned my addr 172.21.15.49:0/3128552758 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.282+0000 7f21dbfff640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1022b0 0x7f21dc103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21dc1018f0 0x7f21dc102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:06.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1022b0 msgr2=0x7f21dc103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1022b0 0x7f21dc103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21dc1018f0 msgr2=0x7f21dc102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21dc1018f0 0x7f21dc102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc103ad0 con 0x7f21dc100700 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21dbfff640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21dc1022b0 0x7f21dc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21db7fe640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 0x7f21dc104640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e9c0 tx=0x7f21cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00cd80 con 0x7f21dc100700 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1a1bf0 con 0x7f21dc100700 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21cc0035f0 con 0x7f21dc100700 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc010690 con 0x7f21dc100700 2024-03-16T08:17:06.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.283+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1a2110 con 0x7f21dc100700 2024-03-16T08:17:06.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.284+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc1a22e0 con 0x7f21dc100700 2024-03-16T08:17:06.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.286+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f21cc0037a0 con 0x7f21dc100700 2024-03-16T08:17:06.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.286+0000 7f21d8ff9640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:06.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.286+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:06.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.286+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8190+0+0 (secure 0 0 0) 0x7f21cc0cee90 con 0x7f21dc100700 2024-03-16T08:17:06.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.286+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:17:06.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.294+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21cc09bf60 con 0x7f21dc100700 2024-03-16T08:17:06.457 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:06.096+0000 7fa8d7217200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:17:06.457 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:06.197+0000 7fa8d7217200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:17:06.458 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:06.086+0000 7f34a6d22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:17:06.458 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:06.181+0000 7f34a6d22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:17:06.458 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:06.136+0000 7f24c1b66200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T08:17:06.458 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:06.231+0000 7f24c1b66200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T08:17:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.487+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.487+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:17:06.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.503+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f21dc0fddb0 con 0x7f21c40a28f0 2024-03-16T08:17:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-mon[36471]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T08:17:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-mon[36471]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-03-16T08:17:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/441193339' entity='client.admin' 2024-03-16T08:17:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-mon[36471]: from='client.? ' entity='client.admin' 2024-03-16T08:17:06.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-mon[31942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T08:17:06.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-mon[31942]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-03-16T08:17:06.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/441193339' entity='client.admin' 2024-03-16T08:17:06.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-mon[31942]: from='client.? ' entity='client.admin' 2024-03-16T08:17:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-mon[36561]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T08:17:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-mon[36561]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-03-16T08:17:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/441193339' entity='client.admin' 2024-03-16T08:17:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-mon[36561]: from='client.? ' entity='client.admin' 2024-03-16T08:17:06.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.888+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:06.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:06.888+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T08:17:07.225 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:06.942+0000 7f34a6d22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:17:07.226 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.065+0000 7f34a6d22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:17:07.226 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:06.880+0000 7fa8d7217200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:17:07.226 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:06.977+0000 7fa8d7217200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:17:07.226 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.125+0000 7fa8d7217200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:17:07.245 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:06.883+0000 7f24c1b66200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T08:17:07.245 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:06.977+0000 7f24c1b66200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T08:17:07.245 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.131+0000 7f24c1b66200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:17:07.246 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.244+0000 7f24c1b66200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:17:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:17:07.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.225+0000 7fa8d7217200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:17:07.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.313+0000 7fa8d7217200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:17:07.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:17:07.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.345+0000 7f24c1b66200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:17:07.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:07.689+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:07.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:07.689+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T08:17:07.707 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:17:07.707 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.240+0000 7f34a6d22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T08:17:07.707 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.349+0000 7f34a6d22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T08:17:07.707 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.434+0000 7f34a6d22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T08:17:07.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.590+0000 7fa8d7217200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:17:07.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.706+0000 7fa8d7217200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:07.897 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.796+0000 7fa8d7217200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:17:07.922 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.625+0000 7f24c1b66200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:17:07.923 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.737+0000 7f24c1b66200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:07.923 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.825+0000 7f24c1b66200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:17:07.975 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.706+0000 7f34a6d22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T08:17:07.975 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.825+0000 7f34a6d22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:07.975 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:07.913+0000 7f34a6d22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T08:17:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:07.896+0000 7fa8d7217200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:17:08.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:08 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:08.080+0000 7fa8d7217200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:17:08.360 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:07.922+0000 7f24c1b66200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:17:08.360 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:08.100+0000 7f24c1b66200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:17:08.475 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:08.010+0000 7f34a6d22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T08:17:08.476 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:08.189+0000 7f34a6d22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T08:17:09.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:09.290+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:09.290+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T08:17:09.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:09 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:09.109+0000 7fa8d7217200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:17:09.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:09 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:09.249+0000 7fa8d7217200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:17:09.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:09.195+0000 7f24c1b66200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:17:09.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:09.344+0000 7f24c1b66200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:17:09.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:09.250+0000 7f34a6d22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T08:17:09.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:09.396+0000 7f34a6d22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T08:17:09.902 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:09 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:09.599+0000 7fa8d7217200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:17:09.902 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:09 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:09.694+0000 7fa8d7217200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:17:09.903 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:09 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:09.902+0000 7fa8d7217200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:10.079 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:09.724+0000 7f24c1b66200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:17:10.079 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:09.837+0000 7f24c1b66200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:17:10.079 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:10.078+0000 7f24c1b66200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:10.169 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:09.812+0000 7f34a6d22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T08:17:10.169 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:09.929+0000 7f34a6d22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T08:17:10.169 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:10.168+0000 7f34a6d22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:10.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:10.099+0000 7fa8d7217200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:17:10.360 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:10.315+0000 7f24c1b66200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:17:10.676 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:10.405+0000 7fa8d7217200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:10.725 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:10.439+0000 7f34a6d22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T08:17:10.927 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:10.676+0000 7fa8d7217200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:10.971 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:10.659+0000 7f24c1b66200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:11.074 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:10.805+0000 7f34a6d22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T08:17:11.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:10.926+0000 7fa8d7217200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:17:11.288 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:10.970+0000 7f24c1b66200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:11.475 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:11.073+0000 7f34a6d22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T08:17:11.475 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:11.315+0000 7f34a6d22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:17:11.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:11.287+0000 7f24c1b66200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T08:17:12.037 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:11 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:11.781+0000 7fa8d7217200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:17:12.475 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:12.190+0000 7f34a6d22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:17:12.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:12.494+0000 7f21daffd640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2256169095 2024-03-16T08:17:12.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:12.494+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-16T08:17:12.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:12.171+0000 7f24c1b66200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T08:17:12.719 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.383+0000 7fa8d7217200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:17:12.720 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.488+0000 7fa8d7217200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:17:12.720 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.719+0000 7fa8d7217200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:17:12.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:12.742+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 20) ==== 136975+0+0 (secure 0 0 0) 0x7f21cc097ed0 con 0x7f21dc100700 2024-03-16T08:17:12.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:12.743+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 msgr2=0x7f21c40a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:17:12.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:12.743+0000 7f21d8ff9640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/2256169095,v1:172.21.15.49:6801/2256169095] conn(0x7f21c40a28f0 0x7f21c40a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:12.993 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: Active manager daemon x restarted 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: Activating manager daemon x 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: mgrmap e20: x(active, starting, since 0.0124578s), standbys: y, z 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:17:12.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:17:12.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:17:12.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-mon[31942]: Manager daemon x is now available 2024-03-16T08:17:13.110 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:12.784+0000 7f24c1b66200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:17:13.110 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:12.887+0000 7f24c1b66200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: Active manager daemon x restarted 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: Activating manager daemon x 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: mgrmap e20: x(active, starting, since 0.0124578s), standbys: y, z 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:17:13.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:17:13.112 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:17:13.113 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:12 smithi129 ceph-mon[36561]: Manager daemon x is now available 2024-03-16T08:17:13.178 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:12.782+0000 7f34a6d22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T08:17:13.178 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:12.885+0000 7f34a6d22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T08:17:13.178 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: Active manager daemon x restarted 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: Activating manager daemon x 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: mgrmap e20: x(active, starting, since 0.0124578s), standbys: y, z 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T08:17:13.179 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T08:17:13.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T08:17:13.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T08:17:13.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:17:13.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:17:13.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:17:13.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:12 smithi154 ceph-mon[36471]: Manager daemon x is now available 2024-03-16T08:17:13.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.991+0000 7fa839bec700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.287 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.991+0000 7fa839bec700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.991+0000 7fa839bec700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.991+0000 7fa839bec700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.991+0000 7fa839bec700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.992+0000 7fa836be6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.992+0000 7fa836be6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.992+0000 7fa836be6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.992+0000 7fa836be6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.288 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:17:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:17:12.992+0000 7fa836be6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:17:13.475 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z[38364]: 2024-03-16T08:17:13.176+0000 7f34a6d22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:17:13.610 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y[37981]: 2024-03-16T08:17:13.144+0000 7f24c1b66200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T08:17:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.749+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f21cc02c470 con 0x7f21dc100700 2024-03-16T08:17:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.750+0000 7f21d8ff9640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21c40516f0 0x7f21c4051ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.750+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f21cc017e80 con 0x7f21c40516f0 2024-03-16T08:17:13.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.753+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21c40516f0 0x7f21c4051ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.756+0000 7f21daffd640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21c40516f0 0x7f21c4051ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1043b0 tx=0x7f21d0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:13.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.828+0000 7f21d8ff9640 1 -- 172.21.15.49:0/3128552758 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f21cc017e80 con 0x7f21c40516f0 2024-03-16T08:17:13.829 INFO:teuthology.orchestra.run.smithi049.stdout:Schedule set for path / 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.830+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21c40516f0 msgr2=0x7f21c4051ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.830+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21c40516f0 0x7f21c4051ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1043b0 tx=0x7f21d0005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.830+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 msgr2=0x7f21dc104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.830+0000 7f21e1f7e640 1 --2- 172.21.15.49:0/3128552758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21dc100700 0x7f21dc104640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e9c0 tx=0x7f21cc00ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.830+0000 7f21dbfff640 1 -- 172.21.15.49:0/3128552758 reap_dead start 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.831+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 shutdown_connections 2024-03-16T08:17:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.831+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 >> 172.21.15.49:0/3128552758 conn(0x7f21dc0fbc70 msgr2=0x7f21dc0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.831+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 shutdown_connections 2024-03-16T08:17:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.831+0000 7f21e1f7e640 1 -- 172.21.15.49:0/3128552758 wait complete. 2024-03-16T08:17:13.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-16T08:17:13.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- 172.21.15.49:0/2675921912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774100700 msgr2=0x7f5774100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:13.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 --2- 172.21.15.49:0/2675921912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774100700 0x7f5774100b20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f576c009900 tx=0x7f576c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:13.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- 172.21.15.49:0/2675921912 shutdown_connections 2024-03-16T08:17:13.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 --2- 172.21.15.49:0/2675921912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 0x7f5774108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:13.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 --2- 172.21.15.49:0/2675921912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57741018f0 0x7f5774101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 --2- 172.21.15.49:0/2675921912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774100700 0x7f5774100b20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- 172.21.15.49:0/2675921912 >> 172.21.15.49:0/2675921912 conn(0x7f57740fbc70 msgr2=0x7f57740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- 172.21.15.49:0/2675921912 shutdown_connections 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- 172.21.15.49:0/2675921912 wait complete. 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 Processor -- start 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.954+0000 7f577af7e640 1 -- start start 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 0x7f5774199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 0x7f57741a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c350 con 0x7f57741018f0 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c1d0 con 0x7f57741022b0 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577af7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c4d0 con 0x7f5774100700 2024-03-16T08:17:13.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5773fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 0x7f5774199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5773fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34092/0 (socket says 172.21.15.49:34092) 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5773fff640 1 -- 172.21.15.49:0/2472676074 learned_addr learned my addr 172.21.15.49:0/2472676074 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f577951b640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 0x7f57741a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 -- 172.21.15.49:0/2472676074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 msgr2=0x7f57741a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 0x7f57741a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 -- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 msgr2=0x7f577419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 -- 172.21.15.49:0/2472676074 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57741a18b0 con 0x7f5774100700 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5773fff640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5778d1a640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 0x7f5774199cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f576c02c170 tx=0x7f576c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.955+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c041070 con 0x7f5774100700 2024-03-16T08:17:13.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.956+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c004450 con 0x7f5774100700 2024-03-16T08:17:13.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.956+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57741a1b40 con 0x7f5774100700 2024-03-16T08:17:13.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.956+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c003e50 con 0x7f5774100700 2024-03-16T08:17:13.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.956+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a2000 con 0x7f5774100700 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.957+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57741a21d0 con 0x7f5774100700 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.958+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f576c044070 con 0x7f5774100700 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.959+0000 7f5771ffb640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 0x7f57600a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.959+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f576c0f03b0 con 0x7f5774100700 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.959+0000 7f5773fff640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 0x7f57600a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.959+0000 7f5773fff640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 0x7f57600a4d10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f577419b170 tx=0x7f5764009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:13.962+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f576c0bd020 con 0x7f5774100700 2024-03-16T08:17:14.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: Standby manager daemon y restarted 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: Standby manager daemon y started 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:17:14.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: Standby manager daemon z restarted 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: Standby manager daemon z started 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: [16/Mar/2024:08:17:13] ENGINE Bus STARTING 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: [16/Mar/2024:08:17:13] ENGINE Serving on http://172.21.15.49:8765 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: [16/Mar/2024:08:17:13] ENGINE Serving on https://172.21.15.49:7150 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: [16/Mar/2024:08:17:13] ENGINE Bus STARTED 2024-03-16T08:17:14.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:13 smithi049 ceph-mon[31942]: mgrmap e21: x(active, since 1.01776s), standbys: y, z 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.044 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: Standby manager daemon y restarted 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: Standby manager daemon y started 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: Standby manager daemon z restarted 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: Standby manager daemon z started 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: [16/Mar/2024:08:17:13] ENGINE Bus STARTING 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: [16/Mar/2024:08:17:13] ENGINE Serving on http://172.21.15.49:8765 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: [16/Mar/2024:08:17:13] ENGINE Serving on https://172.21.15.49:7150 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: [16/Mar/2024:08:17:13] ENGINE Bus STARTED 2024-03-16T08:17:14.045 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:13 smithi154 ceph-mon[36471]: mgrmap e21: x(active, since 1.01776s), standbys: y, z 2024-03-16T08:17:14.047 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.047 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: Standby manager daemon y restarted 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: Standby manager daemon y started 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.129:0/4222698462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: Standby manager daemon z restarted 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: Standby manager daemon z started 2024-03-16T08:17:14.048 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: from='mgr.? 172.21.15.154:0/4076784903' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: [16/Mar/2024:08:17:13] ENGINE Bus STARTING 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: [16/Mar/2024:08:17:13] ENGINE Serving on http://172.21.15.49:8765 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: [16/Mar/2024:08:17:13] ENGINE Serving on https://172.21.15.49:7150 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: [16/Mar/2024:08:17:13] ENGINE Bus STARTED 2024-03-16T08:17:14.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:13 smithi129 ceph-mon[36561]: mgrmap e21: x(active, since 1.01776s), standbys: y, z 2024-03-16T08:17:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.173+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f57740fddd0 con 0x7f57600a2850 2024-03-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.228+0000 7f5771ffb640 1 -- 172.21.15.49:0/2472676074 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f57740fddd0 con 0x7f57600a2850 2024-03-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi049.stdout:Retention added to path / 2024-03-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 msgr2=0x7f57600a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 0x7f57600a4d10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f577419b170 tx=0x7f5764009500 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 msgr2=0x7f5774199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 0x7f5774199cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f576c02c170 tx=0x7f576c02c650 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 shutdown_connections 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57600a2850 0x7f57600a4d10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57741022b0 0x7f57741a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57741018f0 0x7f577419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 --2- 172.21.15.49:0/2472676074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5774100700 0x7f5774199cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 >> 172.21.15.49:0/2472676074 conn(0x7f57740fbc70 msgr2=0x7f57740fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 shutdown_connections 2024-03-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.230+0000 7f577af7e640 1 -- 172.21.15.49:0/2472676074 wait complete. 2024-03-16T08:17:14.242 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 -- 172.21.15.49:0/2456170796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c100750 msgr2=0x7f6b1c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/2456170796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c100750 0x7f6b1c100b70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009900 tx=0x7f6b1002bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 -- 172.21.15.49:0/2456170796 shutdown_connections 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/2456170796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c102300 0x7f6b1c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/2456170796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c101940 0x7f6b1c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/2456170796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c100750 0x7f6b1c100b70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 -- 172.21.15.49:0/2456170796 >> 172.21.15.49:0/2456170796 conn(0x7f6b1c0fbcc0 msgr2=0x7f6b1c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 -- 172.21.15.49:0/2456170796 shutdown_connections 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.352+0000 7f6b21fb9640 1 -- 172.21.15.49:0/2456170796 wait complete. 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 Processor -- start 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 -- start start 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 0x7f6b1c06ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 0x7f6b1c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c10c3d0 con 0x7f6b1c102300 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c10c250 con 0x7f6b1c100750 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b21fb9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c10c550 con 0x7f6b1c101940 2024-03-16T08:17:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 0x7f6b1c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 0x7f6b1c06ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36838/0 (socket says 172.21.15.49:36838) 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 0x7f6b1c06ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:44866/0 (socket says 172.21.15.49:44866) 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1affd640 1 -- 172.21.15.49:0/882176113 learned_addr learned my addr 172.21.15.49:0/882176113 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.353+0000 7f6b1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 0x7f6b1c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34112/0 (socket says 172.21.15.49:34112) 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 -- 172.21.15.49:0/882176113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 msgr2=0x7f6b1c06ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 0x7f6b1c06ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 -- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 msgr2=0x7f6b1c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 0x7f6b1c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 -- 172.21.15.49:0/882176113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c19f700 con 0x7f6b1c101940 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b1affd640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cd80 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c010910 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c196600 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c018990 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.354+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c196b50 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.355+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c107a10 con 0x7f6b1c101940 2024-03-16T08:17:14.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.356+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f6b0c010ac0 con 0x7f6b1c101940 2024-03-16T08:17:14.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.357+0000 7f6b18ff9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 0x7f6b040a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.357+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 0x7f6b040a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.357+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6b0c0cebd0 con 0x7f6b1c101940 2024-03-16T08:17:14.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.357+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 0x7f6b040a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000c00 tx=0x7f6b1003e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.360+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b0c09b840 con 0x7f6b1c101940 2024-03-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.572+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6b1c061c20 con 0x7f6b040a26b0 2024-03-16T08:17:14.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.580+0000 7f6b18ff9640 1 -- 172.21.15.49:0/882176113 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f6b1c061c20 con 0x7f6b040a26b0 2024-03-16T08:17:14.581 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-16T00:00:00", "created": "2024-03-16T08:17:13", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 msgr2=0x7f6b040a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 0x7f6b040a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000c00 tx=0x7f6b1003e040 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 msgr2=0x7f6b1c19c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 shutdown_connections 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b040a26b0 0x7f6b040a4b70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b1c102300 0x7f6b1c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1c101940 0x7f6b1c19c7d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 --2- 172.21.15.49:0/882176113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b1c100750 0x7f6b1c06ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.582+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 >> 172.21.15.49:0/882176113 conn(0x7f6b1c0fbcc0 msgr2=0x7f6b1c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.583+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 shutdown_connections 2024-03-16T08:17:14.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.583+0000 7f6b21fb9640 1 -- 172.21.15.49:0/882176113 wait complete. 2024-03-16T08:17:14.595 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-16T08:17:14.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 -- 172.21.15.49:0/3239410485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288102380 msgr2=0x7f2288104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 --2- 172.21.15.49:0/3239410485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288102380 0x7f2288104770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f228402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 -- 172.21.15.49:0/3239410485 shutdown_connections 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 --2- 172.21.15.49:0/3239410485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 0x7f22881099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 --2- 172.21.15.49:0/3239410485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288104cb0 0x7f22881070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 --2- 172.21.15.49:0/3239410485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288102380 0x7f2288104770 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 -- 172.21.15.49:0/3239410485 >> 172.21.15.49:0/3239410485 conn(0x7f22880fbcb0 msgr2=0x7f22880fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 -- 172.21.15.49:0/3239410485 shutdown_connections 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.711+0000 7f228f7e6640 1 -- 172.21.15.49:0/3239410485 wait complete. 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 Processor -- start 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 -- start start 2024-03-16T08:17:14.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 0x7f2288100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f228810d4f0 con 0x7f2288104cb0 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f228810d370 con 0x7f22881075e0 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228f7e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f228810d670 con 0x7f2288102380 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228cd81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228d582640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228d582640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36852/0 (socket says 172.21.15.49:36852) 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228cd81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34128/0 (socket says 172.21.15.49:34128) 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228d582640 1 -- 172.21.15.49:0/446879704 learned_addr learned my addr 172.21.15.49:0/446879704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228dd83640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 0x7f2288100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228cd81640 1 -- 172.21.15.49:0/446879704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 msgr2=0x7f2288101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.712+0000 7f228cd81640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228cd81640 1 -- 172.21.15.49:0/446879704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 msgr2=0x7f2288100870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228cd81640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 0x7f2288100870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228cd81640 1 -- 172.21.15.49:0/446879704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288101100 con 0x7f2288104cb0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228d582640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228cd81640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f227800e9c0 tx=0x7f227800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227800ccc0 con 0x7f2288104cb0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881a1c10 con 0x7f2288104cb0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2278005020 con 0x7f2288104cb0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278004a00 con 0x7f2288104cb0 2024-03-16T08:17:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.713+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a2160 con 0x7f2288104cb0 2024-03-16T08:17:14.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.714+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880fd7a0 con 0x7f2288104cb0 2024-03-16T08:17:14.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.716+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f22780051d0 con 0x7f2288104cb0 2024-03-16T08:17:14.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.716+0000 7f227e7fc640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 0x7f22740a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:14.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.717+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f22780ce6f0 con 0x7f2288104cb0 2024-03-16T08:17:14.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.717+0000 7f228d582640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 0x7f22740a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:14.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.717+0000 7f228d582640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 0x7f22740a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f228402c880 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:14.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.721+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f227809ba40 con 0x7f2288104cb0 2024-03-16T08:17:14.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.953+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2288061c20 con 0x7f22740a2730 2024-03-16T08:17:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.961+0000 7f227e7fc640 1 -- 172.21.15.49:0/446879704 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2288061c20 con 0x7f22740a2730 2024-03-16T08:17:14.962 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 msgr2=0x7f22740a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 0x7f22740a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f228402c880 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 msgr2=0x7f2288100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f227800e9c0 tx=0x7f227800ee80 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 shutdown_connections 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f22740a2730 0x7f22740a4bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22881075e0 0x7f2288100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288104cb0 0x7f2288100330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 --2- 172.21.15.49:0/446879704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2288102380 0x7f2288101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 >> 172.21.15.49:0/446879704 conn(0x7f22880fbcb0 msgr2=0x7f2288102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.963+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 shutdown_connections 2024-03-16T08:17:14.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:14.964+0000 7f228f7e6640 1 -- 172.21.15.49:0/446879704 wait complete. 2024-03-16T08:17:14.977 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:14 smithi049 ceph-mon[31942]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:14.977 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:14 smithi049 ceph-mon[31942]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:14.977 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:14 smithi049 ceph-mon[31942]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:14.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-16T08:17:15.010 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-16T08:17:15.021 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-16T08:17:15.037 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:17:15.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-16T08:17:15.053 INFO:teuthology.run_tasks:Running task workunit... 2024-03-16T08:17:15.068 INFO:tasks.workunit:Pulling workunits from ref 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:17:15.068 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-16T08:17:15.068 INFO:tasks.workunit:timeout=3h 2024-03-16T08:17:15.068 INFO:tasks.workunit:cleanup=True 2024-03-16T08:17:15.068 DEBUG:teuthology.orchestra.run.smithi049:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-16T08:17:15.084 INFO:teuthology.orchestra.run.smithi049.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-16T08:17:15.084 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-16T08:17:15.084 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-16T08:17:15.084 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:17:15.084 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-16T08:17:15.085 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-16 08:15:09.019005780 +0000 2024-03-16T08:17:15.085 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-16 08:15:09.019005780 +0000 2024-03-16T08:17:15.085 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-16 08:16:57.607427444 +0000 2024-03-16T08:17:15.085 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-16 08:15:09.019005780 +0000 2024-03-16T08:17:15.085 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-16T08:17:15.085 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-16T08:17:15.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:14 smithi129 ceph-mon[36561]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:15.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:14 smithi129 ceph-mon[36561]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:15.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:14 smithi129 ceph-mon[36561]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:15.164 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:17:15.220 INFO:tasks.workunit.client.0.smithi049.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-16T08:17:15.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:14 smithi154 ceph-mon[36471]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:15.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:14 smithi154 ceph-mon[36471]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:15.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:14 smithi154 ceph-mon[36471]: from='client.25375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:15 smithi154 ceph-mon[36471]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-03-16T08:17:16.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:15 smithi049 ceph-mon[31942]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-03-16T08:17:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:15 smithi129 ceph-mon[36561]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-03-16T08:17:17.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.362 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.363 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.364 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.365 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:17:17.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:17:17.368 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:17:17.368 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:17 smithi129 ceph-mon[36561]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.478 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:17:17.479 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:17 smithi154 ceph-mon[36471]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:17:17.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:17.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:17.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-16T08:17:17.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Updating smithi129:/etc/ceph/ceph.conf 2024-03-16T08:17:17.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Updating smithi154:/etc/ceph/ceph.conf 2024-03-16T08:17:17.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:17 smithi049 ceph-mon[31942]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-16T08:17:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:18 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:18.539 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:18 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: Updating smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:17:18.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:18 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:17:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:20 smithi154 ceph-mon[36471]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-16T08:17:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:20 smithi049 ceph-mon[31942]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-16T08:17:20.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:20 smithi129 ceph-mon[36561]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-16T08:17:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:22 smithi154 ceph-mon[36471]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-16T08:17:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:22 smithi049 ceph-mon[31942]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-16T08:17:22.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:22 smithi129 ceph-mon[36561]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-16T08:17:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:24 smithi154 ceph-mon[36471]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-16T08:17:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:24 smithi049 ceph-mon[31942]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-16T08:17:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:24 smithi129 ceph-mon[36561]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-16T08:17:26.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:26 smithi129 ceph-mon[36561]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-16T08:17:26.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:26 smithi154 ceph-mon[36471]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-16T08:17:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:26 smithi049 ceph-mon[31942]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-16T08:17:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 -- 172.21.15.49:0/93081939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834105680 msgr2=0x7f5834105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 --2- 172.21.15.49:0/93081939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834105680 0x7f5834105a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5824009900 tx=0x7f582402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 -- 172.21.15.49:0/93081939 shutdown_connections 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 --2- 172.21.15.49:0/93081939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58341012f0 0x7f583410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 --2- 172.21.15.49:0/93081939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5834100930 0x7f5834100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 --2- 172.21.15.49:0/93081939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834105680 0x7f5834105a60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.200+0000 7f583960b640 1 -- 172.21.15.49:0/93081939 >> 172.21.15.49:0/93081939 conn(0x7f5834077530 msgr2=0x7f5834077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.201+0000 7f583960b640 1 -- 172.21.15.49:0/93081939 shutdown_connections 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.201+0000 7f583960b640 1 -- 172.21.15.49:0/93081939 wait complete. 2024-03-16T08:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.201+0000 7f583960b640 1 Processor -- start 2024-03-16T08:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.201+0000 7f583960b640 1 -- start start 2024-03-16T08:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 0x7f5834198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5834105680 0x7f583419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110b60 con 0x7f5834100930 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58341109e0 con 0x7f58341012f0 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f583960b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110ce0 con 0x7f5834105680 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46848/0 (socket says 172.21.15.49:46848) 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f5832ffd640 1 -- 172.21.15.49:0/3620151146 learned_addr learned my addr 172.21.15.49:0/3620151146 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 0x7f5834198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58337fe640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5834105680 0x7f583419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 -- 172.21.15.49:0/3620151146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5834105680 msgr2=0x7f583419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5834105680 0x7f583419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 -- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 msgr2=0x7f5834198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f58327fc640 1 -- 172.21.15.49:0/3620151146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419d020 con 0x7f58341012f0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.202+0000 7f5832ffd640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f58327fc640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 0x7f5834198570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f582800e9f0 tx=0x7f582800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f581bfff640 1 -- 172.21.15.49:0/3620151146 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800ccc0 con 0x7f58341012f0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f581bfff640 1 -- 172.21.15.49:0/3620151146 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5828010910 con 0x7f58341012f0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419d310 con 0x7f58341012f0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f581bfff640 1 -- 172.21.15.49:0/3620151146 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828018960 con 0x7f58341012f0 2024-03-16T08:17:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.203+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a4ba0 con 0x7f58341012f0 2024-03-16T08:17:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.205+0000 7f581bfff640 1 -- 172.21.15.49:0/3620151146 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f5828010ac0 con 0x7f58341012f0 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 Processor -- start 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f581bfff640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 0x7f581c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 -- start start 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58337fe640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 0x7f581c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f581bfff640 1 -- 172.21.15.49:0/3620151146 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f58280cf3b0 con 0x7f58341012f0 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 0x55db8b48ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 0x55db8b49d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55db8b3f1a40 con 0x55db8b48d410 2024-03-16T08:17:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db8b39a920 con 0x55db8b48caf0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f583c1fc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55db8b3edae0 con 0x55db8b49ac30 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58327fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58327fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46858/0 (socket says 172.21.15.49:46858) 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58327fc640 1 -- 172.21.15.49:0/723812175 learned_addr learned my addr 172.21.15.49:0/723812175 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f5832ffd640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 0x55db8b48ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58337fe640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 0x55db8b49d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.206+0000 7f58337fe640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 0x7f581c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f582c005fd0 tx=0x7f582c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 -- 172.21.15.49:0/723812175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 msgr2=0x55db8b49d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 0x55db8b49d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 -- 172.21.15.49:0/723812175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 msgr2=0x55db8b49a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 -- 172.21.15.49:0/723812175 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8b3b2460 con 0x55db8b48caf0 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f58337fe640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 0x55db8b49d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f58327fc640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f5832ffd640 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 0x55db8b48ced0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f582402efd0 tx=0x7f5824009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da905ab28c26ffc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f58197fa640 1 -- 172.21.15.49:0/723812175 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824042070 con 0x55db8b48caf0 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f58197fa640 1 -- 172.21.15.49:0/723812175 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5824004a70 con 0x55db8b48caf0 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.207+0000 7f58197fa640 1 -- 172.21.15.49:0/723812175 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824005160 con 0x55db8b48caf0 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- 172.21.15.49:0/723812175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 msgr2=0x55db8b48ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 0x55db8b48ced0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f582402efd0 tx=0x7f5824009560 comp rx=0 tx=0).stop 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- 172.21.15.49:0/723812175 shutdown_connections 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b49ac30 0x55db8b49d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b48d410 0x55db8b49a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- 172.21.15.49:0/723812175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48caf0 0x55db8b48ced0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- 172.21.15.49:0/723812175 >> 172.21.15.49:0/723812175 conn(0x55db8b487a50 msgr2=0x55db8b487e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- 172.21.15.49:0/723812175 shutdown_connections 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- 172.21.15.49:0/723812175 wait complete. 2024-03-16T08:17:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- start start 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 Processor -- start 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55db8b49d800 con 0x55db8b49ac30 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db8b3add70 con 0x55db8b48d410 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55db8b49d980 con 0x55db8b48caf0 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.208+0000 7f583c1fc740 10 client.0 _ll_register_callbacks cb 0x55db8b105a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46870/0 (socket says 172.21.15.49:46870) 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41240/0 (socket says 172.21.15.49:41240) 2024-03-16T08:17:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5832ffd640 1 -- 172.21.15.49:0/2455668072 learned_addr learned my addr 172.21.15.49:0/2455668072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39466/0 (socket says 172.21.15.49:39466) 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 -- 172.21.15.49:0/2455668072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 msgr2=0x55db8b497c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 -- 172.21.15.49:0/2455668072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 msgr2=0x55db8b498710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 -- 172.21.15.49:0/2455668072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8b106210 con 0x55db8b48d410 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5832ffd640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58337fe640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f58327fc640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f582809b660 tx=0x7f582809baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b5c267010630c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5816ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280b4060 con 0x55db8b48d410 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f582809cde0 con 0x55db8b48d410 2024-03-16T08:17:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824004a70 con 0x55db8b48d410 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f58280ccf10 con 0x55db8b48d410 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f583c1fc740 10 client.25409 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.209+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55db8b1d2620 con 0x55db8b48d410 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5824004a70 con 0x55db8b48d410 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5828018b00 con 0x55db8b48d410 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 10 client.25409 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 10 client.25409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 20 client.25409 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b4aadc0 0x55db8b4ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 4 client.25409 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db8b3fef80 con 0x55db8b4aadc0 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f5832ffd640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b4aadc0 0x55db8b4ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f5816ff5640 10 client.25409 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.210+0000 7f5832ffd640 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b4aadc0 0x55db8b4ad1f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df009d4dc6c2d1d server_cookie=208495b2beee41c0 in_seq=0 out_seq=0 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f5816ff5640 1 -- 172.21.15.49:0/2455668072 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db8b3fef80 con 0x55db8b4aadc0 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f5816ff5640 10 client.25409 handle_command_reply: tid=0 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 client.25409 shutdown 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 msgr2=0x55db8b4981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f582809b660 tx=0x7f582809baa0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 shutdown_connections 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b4aadc0 0x55db8b4ad1f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b498710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4981d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 --2- 172.21.15.49:0/2455668072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b497c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f5816ff5640 0 client.25409 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 >> 172.21.15.49:0/2455668072 conn(0x55db8b487a50 msgr2=0x55db8b4897a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 shutdown_connections 2024-03-16T08:17:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.211+0000 7f583c1fc740 1 -- 172.21.15.49:0/2455668072 wait complete. 2024-03-16T08:17:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.212+0000 7f583c1fc740 20 client.25409 trim_cache size 0 max 16384 2024-03-16T08:17:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 Processor -- start 2024-03-16T08:17:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 -- start start 2024-03-16T08:17:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b49a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58341ad8a0 con 0x55db8b49ac30 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58341ab170 con 0x55db8b48d410 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.226+0000 7f583c1fc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58341a8fa0 con 0x55db8b48caf0 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46882/0 (socket says 172.21.15.49:46882) 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 -- 172.21.15.49:0/3132255153 learned_addr learned my addr 172.21.15.49:0/3132255153 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f5832ffd640 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b49a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58327fc640 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 -- 172.21.15.49:0/3132255153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 msgr2=0x55db8b49a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b49a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 -- 172.21.15.49:0/3132255153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 msgr2=0x55db8b4a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 -- 172.21.15.49:0/3132255153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8b48c410 con 0x55db8b49ac30 2024-03-16T08:17:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f58337fe640 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f582c01cac0 tx=0x7f582c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7fce514d893eb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f5818ff9640 1 -- 172.21.15.49:0/3132255153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c02b070 con 0x55db8b49ac30 2024-03-16T08:17:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f5818ff9640 1 -- 172.21.15.49:0/3132255153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c003970 con 0x55db8b49ac30 2024-03-16T08:17:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.227+0000 7f5818ff9640 1 -- 172.21.15.49:0/3132255153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0046f0 con 0x55db8b49ac30 2024-03-16T08:17:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- 172.21.15.49:0/3132255153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 msgr2=0x55db8b4b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f582c01cac0 tx=0x7f582c019000 comp rx=0 tx=0).stop 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- 172.21.15.49:0/3132255153 shutdown_connections 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b4b2b00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 --2- 172.21.15.49:0/3132255153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b49a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- 172.21.15.49:0/3132255153 >> 172.21.15.49:0/3132255153 conn(0x55db8b487a50 msgr2=0x55db8b482af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- 172.21.15.49:0/3132255153 shutdown_connections 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- 172.21.15.49:0/3132255153 wait complete. 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 -- start start 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.228+0000 7f583c1fc740 1 Processor -- start 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b494b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58341a9630 con 0x55db8b49ac30 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58341aaaa0 con 0x55db8b48d410 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58341a9d00 con 0x55db8b48caf0 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 10 client.0 _ll_register_callbacks cb 0x7f58280cf2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46884/0 (socket says 172.21.15.49:46884) 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 -- 172.21.15.49:0/450038771 learned_addr learned my addr 172.21.15.49:0/450038771 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f5832ffd640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b494b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58327fc640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 -- 172.21.15.49:0/450038771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 msgr2=0x55db8b494b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b494b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 -- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 msgr2=0x55db8b4950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 -- 172.21.15.49:0/450038771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8b510400 con 0x55db8b49ac30 2024-03-16T08:17:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f58337fe640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f582c01bfc0 tx=0x7f582c004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f69badaaf2e762e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f5816ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c02b040 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f582c01bc70 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.229+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c003970 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f582c03d050 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 10 client.15528 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55db8b511710 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c003970 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f582c02e070 con 0x55db8b49ac30 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 10 client.15528 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 20 client.15528 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b511d90 0x55db8b5141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 4 client.15528 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:17:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db8b3f92d0 con 0x55db8b511d90 2024-03-16T08:17:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.230+0000 7f5832ffd640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b511d90 0x55db8b5141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.231+0000 7f5816ff5640 10 client.15528 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.231+0000 7f5832ffd640 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b511d90 0x55db8b5141b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b898779b64a7d5 server_cookie=10da18e4f9c8b422 in_seq=0 out_seq=0 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f5816ff5640 1 -- 172.21.15.49:0/450038771 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db8b3f92d0 con 0x55db8b511d90 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f5816ff5640 10 client.15528 handle_command_reply: tid=0 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 client.15528 shutdown 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 msgr2=0x55db8b495620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f582c01bfc0 tx=0x7f582c004050 comp rx=0 tx=0).stop 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 shutdown_connections 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db8b511d90 0x55db8b5141b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8b49ac30 0x55db8b495620 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db8b48d410 0x55db8b4950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f5816ff5640 0 client.15528 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 --2- 172.21.15.49:0/450038771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db8b48caf0 0x55db8b494b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 >> 172.21.15.49:0/450038771 conn(0x55db8b487a50 msgr2=0x55db8b486320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 shutdown_connections 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 1 -- 172.21.15.49:0/450038771 wait complete. 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.240+0000 7f583c1fc740 20 client.15528 trim_cache size 0 max 16384 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:17:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 msgr2=0x7f581c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 0x7f581c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f582c005fd0 tx=0x7f582c009500 comp rx=0 tx=0).stop 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 msgr2=0x7f5834198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 0x7f5834198570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f582800e9f0 tx=0x7f582800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 shutdown_connections 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f581c0a2ab0 0x7f581c0a4f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5834105680 0x7f583419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58341012f0 0x7f5834198570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 --2- 172.21.15.49:0/3620151146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5834100930 0x7f5834198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 >> 172.21.15.49:0/3620151146 conn(0x7f5834077530 msgr2=0x7f5834069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 shutdown_connections 2024-03-16T08:17:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:28.241+0000 7f583960b640 1 -- 172.21.15.49:0/3620151146 wait complete. 2024-03-16T08:17:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:28 smithi154 ceph-mon[36471]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-16T08:17:28.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:28 smithi049 ceph-mon[31942]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-16T08:17:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:28 smithi129 ceph-mon[36561]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-16T08:17:30.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:30 smithi154 ceph-mon[36471]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-03-16T08:17:30.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:30 smithi049 ceph-mon[31942]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-03-16T08:17:30.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:30 smithi129 ceph-mon[36561]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-03-16T08:17:32.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:32 smithi154 ceph-mon[36471]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 53 op/s 2024-03-16T08:17:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:32 smithi049 ceph-mon[31942]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 53 op/s 2024-03-16T08:17:32.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:32 smithi129 ceph-mon[36561]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 53 op/s 2024-03-16T08:17:34.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:34 smithi154 ceph-mon[36471]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:34.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:34 smithi049 ceph-mon[31942]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:34 smithi129 ceph-mon[36561]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:34.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:17:34.764 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:17:34.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 -- 172.21.15.49:0/1805943429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324102a50 msgr2=0x7fe324102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 --2- 172.21.15.49:0/1805943429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324102a50 0x7fe324102e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe314009900 tx=0x7fe31402bcd0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 -- 172.21.15.49:0/1805943429 shutdown_connections 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 --2- 172.21.15.49:0/1805943429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe32410be20 0x7fe32410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 --2- 172.21.15.49:0/1805943429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe324103370 0x7fe32410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 --2- 172.21.15.49:0/1805943429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324102a50 0x7fe324102e30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 -- 172.21.15.49:0/1805943429 >> 172.21.15.49:0/1805943429 conn(0x7fe324077840 msgr2=0x7fe324077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 -- 172.21.15.49:0/1805943429 shutdown_connections 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.885+0000 7fe329685640 1 -- 172.21.15.49:0/1805943429 wait complete. 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 Processor -- start 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 -- start start 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 0x7fe32419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe324111bf0 con 0x7fe324103370 2024-03-16T08:17:34.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe324111a70 con 0x7fe32410be20 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe329685640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe324111d70 con 0x7fe324102a50 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe3227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe3227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50540/0 (socket says 172.21.15.49:50540) 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:51526/0 (socket says 172.21.15.49:51526) 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 0x7fe32419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.886+0000 7fe3227fc640 1 -- 172.21.15.49:0/4114974 learned_addr learned my addr 172.21.15.49:0/4114974 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 -- 172.21.15.49:0/4114974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 msgr2=0x7fe3241980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 -- 172.21.15.49:0/4114974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 msgr2=0x7fe32419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 0x7fe32419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 -- 172.21.15.49:0/4114974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32419d0a0 con 0x7fe324103370 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3237fe640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 0x7fe32419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe322ffd640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe3227fc640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe30bfff640 1 -- 172.21.15.49:0/4114974 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31800cd80 con 0x7fe324103370 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32419d390 con 0x7fe324103370 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe30bfff640 1 -- 172.21.15.49:0/4114974 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318010910 con 0x7fe324103370 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe30bfff640 1 -- 172.21.15.49:0/4114974 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180188f0 con 0x7fe324103370 2024-03-16T08:17:34.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.887+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3241a4c20 con 0x7fe324103370 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.889+0000 7fe30bfff640 1 -- 172.21.15.49:0/4114974 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fe318010ac0 con 0x7fe324103370 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 Processor -- start 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 -- start start 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e20110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 0x55d2e0e22a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0d7b500 con 0x55d2e0e20650 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe30bfff640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 0x7fe30c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0d23be0 con 0x55d2e0e17a70 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe32c27b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0d775a0 con 0x55d2e0e17150 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe3227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 0x55d2e0e22a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:51528/0 (socket says 172.21.15.49:51528) 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe3237fe640 1 -- 172.21.15.49:0/2890054550 learned_addr learned my addr 172.21.15.49:0/2890054550 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e20110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe3237fe640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 0x7fe30c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.890+0000 7fe30bfff640 1 -- 172.21.15.49:0/4114974 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe3180cf3a0 con 0x7fe324103370 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 -- 172.21.15.49:0/2890054550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 msgr2=0x55d2e0e17530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 -- 172.21.15.49:0/2890054550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 msgr2=0x55d2e0e20110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e20110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 -- 172.21.15.49:0/2890054550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e0d374b0 con 0x55d2e0e20650 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe322ffd640 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e20110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3237fe640 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 0x55d2e0e22a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe3180cca40 tx=0x7fe31809b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85bc225bee361e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3237fe640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 0x7fe30c0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005c40 tx=0x7fe31c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3097fa640 1 -- 172.21.15.49:0/2890054550 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31806dd20 con 0x55d2e0e20650 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3097fa640 1 -- 172.21.15.49:0/2890054550 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31809b950 con 0x55d2e0e20650 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.891+0000 7fe3097fa640 1 -- 172.21.15.49:0/2890054550 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31809bbb0 con 0x55d2e0e20650 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- 172.21.15.49:0/2890054550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 msgr2=0x55d2e0e22a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 0x55d2e0e22a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe3180cca40 tx=0x7fe31809b5c0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- 172.21.15.49:0/2890054550 shutdown_connections 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e20650 0x55d2e0e22a40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e20110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2890054550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- 172.21.15.49:0/2890054550 >> 172.21.15.49:0/2890054550 conn(0x55d2e0e11880 msgr2=0x55d2e0e11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- 172.21.15.49:0/2890054550 shutdown_connections 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- 172.21.15.49:0/2890054550 wait complete. 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- start start 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 Processor -- start 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e20650 0x55d2e0e1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0e277f0 con 0x55d2e0e17150 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0d37160 con 0x55d2e0e17a70 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0e27970 con 0x55d2e0e20650 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.892+0000 7fe32c27b740 10 client.0 _ll_register_callbacks cb 0x55d2e0d1b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:34.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50556/0 (socket says 172.21.15.49:50556) 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 -- 172.21.15.49:0/1938664756 learned_addr learned my addr 172.21.15.49:0/1938664756 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe3237fe640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e20650 0x55d2e0e1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe3227fc640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 -- 172.21.15.49:0/1938664756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e20650 msgr2=0x55d2e0e1e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e20650 0x55d2e0e1e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 -- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 msgr2=0x55d2e0e1dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 -- 172.21.15.49:0/1938664756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e0b597d0 con 0x55d2e0e17150 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe322ffd640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe314009230 tx=0x7fe314005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e50ae956caa4ea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe306ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31402cce0 con 0x55d2e0e17150 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe314004040 con 0x55d2e0e17150 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31809b950 con 0x55d2e0e17150 2024-03-16T08:17:34.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe314004240 con 0x55d2e0e17150 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe32c27b740 10 client.15540 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.893+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2e0a8f3f0 con 0x55d2e0e17150 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31809b950 con 0x55d2e0e17150 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe314003890 con 0x55d2e0e17150 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 10 client.15540 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 20 client.15540 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 4 client.15540 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e0d88a40 con 0x55d2e0e34b70 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe3227fc640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe3227fc640 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e36fc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06f1c3bdb94a61b server_cookie=fa638dbf5253cad4 in_seq=0 out_seq=0 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.894+0000 7fe306ff5640 10 client.15540 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:34.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe306ff5640 1 -- 172.21.15.49:0/1938664756 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d2e0d88a40 con 0x55d2e0e34b70 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe306ff5640 10 client.15540 handle_command_reply: tid=0 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 client.15540 shutdown 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 msgr2=0x55d2e0e1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe314009230 tx=0x7fe314005bf0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 shutdown_connections 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e36fc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e20650 0x55d2e0e1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe306ff5640 0 client.15540 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.895+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.896+0000 7fe32c27b740 1 --2- 172.21.15.49:0/1938664756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e1d760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.896+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 >> 172.21.15.49:0/1938664756 conn(0x55d2e0e11880 msgr2=0x55d2e0e135d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.896+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 shutdown_connections 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.896+0000 7fe32c27b740 1 -- 172.21.15.49:0/1938664756 wait complete. 2024-03-16T08:17:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.896+0000 7fe32c27b740 20 client.15540 trim_cache size 0 max 16384 2024-03-16T08:17:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 Processor -- start 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 -- start start 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e1fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241adac0 con 0x55d2e0e17a70 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241ab390 con 0x55d2e0e20650 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe32c27b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241a91c0 con 0x55d2e0e17150 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.913+0000 7fe3227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50566/0 (socket says 172.21.15.49:50566) 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 -- 172.21.15.49:0/2735629885 learned_addr learned my addr 172.21.15.49:0/2735629885 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe322ffd640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e1fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3237fe640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 -- 172.21.15.49:0/2735629885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 msgr2=0x55d2e0e1fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e1fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 -- 172.21.15.49:0/2735629885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 msgr2=0x55d2e0e18ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e18ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 -- 172.21.15.49:0/2735629885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e0a4f0f0 con 0x55d2e0e17a70 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3237fe640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe3227fc640 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe31809b8e0 tx=0x7fe318086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0c08814dd68272 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe308ff9640 1 -- 172.21.15.49:0/2735629885 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31809b950 con 0x55d2e0e17a70 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe308ff9640 1 -- 172.21.15.49:0/2735629885 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318099c50 con 0x55d2e0e17a70 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.914+0000 7fe308ff9640 1 -- 172.21.15.49:0/2735629885 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180a3bb0 con 0x55d2e0e17a70 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- 172.21.15.49:0/2735629885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 msgr2=0x55d2e0e1c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe31809b8e0 tx=0x7fe318086040 comp rx=0 tx=0).stop 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- 172.21.15.49:0/2735629885 shutdown_connections 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17a70 0x55d2e0e1c6a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 --2- 172.21.15.49:0/2735629885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17150 0x55d2e0e1fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- 172.21.15.49:0/2735629885 >> 172.21.15.49:0/2735629885 conn(0x55d2e0e11880 msgr2=0x55d2e0e13560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- 172.21.15.49:0/2735629885 shutdown_connections 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- 172.21.15.49:0/2735629885 wait complete. 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 -- start start 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.915+0000 7fe32c27b740 1 Processor -- start 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17a70 0x55d2e0e28b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e29050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241a9850 con 0x55d2e0e17150 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241aacc0 con 0x55d2e0e20650 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241a9f20 con 0x55d2e0e17a70 2024-03-16T08:17:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 10 client.0 _ll_register_callbacks cb 0x7fe314009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50574/0 (socket says 172.21.15.49:50574) 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 -- 172.21.15.49:0/275210260 learned_addr learned my addr 172.21.15.49:0/275210260 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe3227fc640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17a70 0x55d2e0e28b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe3237fe640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e29050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 -- 172.21.15.49:0/275210260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17a70 msgr2=0x55d2e0e28b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17a70 0x55d2e0e28b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 -- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 msgr2=0x55d2e0e29050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e29050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 -- 172.21.15.49:0/275210260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e0d429d0 con 0x55d2e0e17150 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe322ffd640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe31402c210 tx=0x7fe314000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861f5400cd0cdec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe306ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314055070 con 0x55d2e0e17150 2024-03-16T08:17:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe31402cae0 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.916+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318099c50 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe314000e60 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 10 client.15546 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2e0e9b100 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318099c50 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe314062030 con 0x55d2e0e17150 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 10 client.15546 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 20 client.15546 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 4 client.15546 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2e0d82d90 con 0x55d2e0e34b70 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.917+0000 7fe3227fc640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe306ff5640 10 client.15546 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe3227fc640 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e9d8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f206346322b3e77 server_cookie=4d00f56e3fdf9161 in_seq=0 out_seq=0 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe306ff5640 1 -- 172.21.15.49:0/275210260 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d2e0d82d90 con 0x55d2e0e34b70 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe306ff5640 10 client.15546 handle_command_reply: tid=0 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 client.15546 shutdown 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 msgr2=0x55d2e0e285d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe31402c210 tx=0x7fe314000bc0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 shutdown_connections 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55d2e0e34b70 0x55d2e0e9d8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2e0e20650 0x55d2e0e29050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d2e0e17a70 0x55d2e0e28b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe306ff5640 0 client.15546 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 --2- 172.21.15.49:0/275210260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2e0e17150 0x55d2e0e285d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 >> 172.21.15.49:0/275210260 conn(0x55d2e0e11880 msgr2=0x55d2e0e10070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 shutdown_connections 2024-03-16T08:17:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 1 -- 172.21.15.49:0/275210260 wait complete. 2024-03-16T08:17:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.918+0000 7fe32c27b740 20 client.15546 trim_cache size 0 max 16384 2024-03-16T08:17:34.926 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:34.926 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:17:34.926 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:17:34.926 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 msgr2=0x7fe30c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 0x7fe30c0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005c40 tx=0x7fe31c009500 comp rx=0 tx=0).stop 2024-03-16T08:17:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 msgr2=0x7fe3241985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 shutdown_connections 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe30c0a2a00 0x7fe30c0a4ec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32410be20 0x7fe32419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe324103370 0x7fe3241985f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 --2- 172.21.15.49:0/4114974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe324102a50 0x7fe3241980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 >> 172.21.15.49:0/4114974 conn(0x7fe324077840 msgr2=0x7fe3240fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 shutdown_connections 2024-03-16T08:17:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:34.919+0000 7fe329685640 1 -- 172.21.15.49:0/4114974 wait complete. 2024-03-16T08:17:34.931 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:17:34.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:17:35.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- 172.21.15.49:0/2365054867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1066f0 msgr2=0x7fa0ec106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 --2- 172.21.15.49:0/2365054867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec106ad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009900 tx=0x7fa0d402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- 172.21.15.49:0/2365054867 shutdown_connections 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 --2- 172.21.15.49:0/2365054867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1010a0 0x7fa0ec10d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 --2- 172.21.15.49:0/2365054867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 --2- 172.21.15.49:0/2365054867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec106ad0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- 172.21.15.49:0/2365054867 >> 172.21.15.49:0/2365054867 conn(0x7fa0ec0777e0 msgr2=0x7fa0ec077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- 172.21.15.49:0/2365054867 shutdown_connections 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- 172.21.15.49:0/2365054867 wait complete. 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 Processor -- start 2024-03-16T08:17:35.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.040+0000 7fa0f264b640 1 -- start start 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1010a0 0x7fa0ec1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec110ac0 con 0x7fa0ec1006e0 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec110940 con 0x7fa0ec1010a0 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f264b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec110c40 con 0x7fa0ec1066f0 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50582/0 (socket says 172.21.15.49:50582) 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 -- 172.21.15.49:0/1072086561 learned_addr learned my addr 172.21.15.49:0/1072086561 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0eb7fe640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1010a0 0x7fa0ec1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0f0be8640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 -- 172.21.15.49:0/1072086561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1066f0 msgr2=0x7fa0ec19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 -- 172.21.15.49:0/1072086561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1010a0 msgr2=0x7fa0ec1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1010a0 0x7fa0ec1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 -- 172.21.15.49:0/1072086561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec19d070 con 0x7fa0ec1006e0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.041+0000 7fa0ebfff640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4005940 tx=0x7fa0d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.042+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4041070 con 0x7fa0ec1006e0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.042+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec19d300 con 0x7fa0ec1006e0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.042+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40054e0 con 0x7fa0ec1006e0 2024-03-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.042+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4003d00 con 0x7fa0ec1006e0 2024-03-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.042+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1a4bf0 con 0x7fa0ec1006e0 2024-03-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.043+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec0685d0 con 0x7fa0ec1006e0 2024-03-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.045+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fa0d4044070 con 0x7fa0ec1006e0 2024-03-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.045+0000 7fa0e97fa640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 0x7fa0d80a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.045+0000 7fa0eb7fe640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 0x7fa0d80a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.045+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa0d40f0d50 con 0x7fa0ec1006e0 2024-03-16T08:17:35.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.046+0000 7fa0eb7fe640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 0x7fa0d80a4ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0ffb20 tx=0x7fa0dc005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.049+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa0d40bd450 con 0x7fa0ec1006e0 2024-03-16T08:17:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.271+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0ec109c60 con 0x7fa0ec1006e0 2024-03-16T08:17:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.272+0000 7fa0e97fa640 1 -- 172.21.15.49:0/1072086561 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fa0d40c14f0 con 0x7fa0ec1006e0 2024-03-16T08:17:35.273 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:35.273 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:17:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:17:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 msgr2=0x7fa0d80a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 0x7fa0d80a4ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0ffb20 tx=0x7fa0dc005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 msgr2=0x7fa0ec198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4005940 tx=0x7fa0d4004270 comp rx=0 tx=0).stop 2024-03-16T08:17:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 shutdown_connections 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa0d80a2a00 0x7fa0d80a4ec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0ec1066f0 0x7fa0ec19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0ec1010a0 0x7fa0ec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 --2- 172.21.15.49:0/1072086561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0ec1006e0 0x7fa0ec198080 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 >> 172.21.15.49:0/1072086561 conn(0x7fa0ec0777e0 msgr2=0x7fa0ec068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 shutdown_connections 2024-03-16T08:17:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.274+0000 7fa0f264b640 1 -- 172.21.15.49:0/1072086561 wait complete. 2024-03-16T08:17:35.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:17:35.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 -- 172.21.15.49:0/2708800135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 msgr2=0x7f6960068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 --2- 172.21.15.49:0/2708800135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f6960068e60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f695c009930 tx=0x7f695c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:35.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 -- 172.21.15.49:0/2708800135 shutdown_connections 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 --2- 172.21.15.49:0/2708800135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69600693a0 0x7f696010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 --2- 172.21.15.49:0/2708800135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f6960068e60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 --2- 172.21.15.49:0/2708800135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960107830 0x7f6960107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 -- 172.21.15.49:0/2708800135 >> 172.21.15.49:0/2708800135 conn(0x7f69600777e0 msgr2=0x7f6960077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 -- 172.21.15.49:0/2708800135 shutdown_connections 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.396+0000 7f69678bf640 1 -- 172.21.15.49:0/2708800135 wait complete. 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 Processor -- start 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 -- start start 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69600693a0 0x7f6960100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6960107830 0x7f6960101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111bd0 con 0x7f69600689e0 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111a50 con 0x7f6960107830 2024-03-16T08:17:35.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f69678bf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111d50 con 0x7f69600693a0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50596/0 (socket says 172.21.15.49:50596) 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 -- 172.21.15.49:0/3176959936 learned_addr learned my addr 172.21.15.49:0/3176959936 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f6964e5a640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69600693a0 0x7f6960100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f6965e5c640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6960107830 0x7f6960101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 -- 172.21.15.49:0/3176959936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69600693a0 msgr2=0x7f6960100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69600693a0 0x7f6960100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 -- 172.21.15.49:0/3176959936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6960107830 msgr2=0x7f6960101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6960107830 0x7f6960101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.397+0000 7f696565b640 1 -- 172.21.15.49:0/3176959936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69601a6510 con 0x7f69600689e0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f696565b640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f695000e9f0 tx=0x7f695000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f69567fc640 1 -- 172.21.15.49:0/3176959936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695000cd80 con 0x7f69600689e0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f69567fc640 1 -- 172.21.15.49:0/3176959936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6950005500 con 0x7f69600689e0 2024-03-16T08:17:35.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69601a6800 con 0x7f69600689e0 2024-03-16T08:17:35.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f69567fc640 1 -- 172.21.15.49:0/3176959936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950010690 con 0x7f69600689e0 2024-03-16T08:17:35.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.398+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a6cd0 con 0x7f69600689e0 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.400+0000 7f69567fc640 1 -- 172.21.15.49:0/3176959936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f69500046c0 con 0x7f69600689e0 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.400+0000 7f696a582740 1 Processor -- start 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 -- start start 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 0x560f96df3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96dff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f96d57490 con 0x560f96df4150 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560f96cffc90 con 0x560f96dfcd30 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696a582740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f96d53530 con 0x560f96df3830 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f69567fc640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 0x7f694c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96dff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50612/0 (socket says 172.21.15.49:50612) 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f696565b640 1 -- 172.21.15.49:0/3026645103 learned_addr learned my addr 172.21.15.49:0/3026645103 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6965e5c640 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 0x560f96df3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6965e5c640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 0x7f694c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f69567fc640 1 -- 172.21.15.49:0/3176959936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f69500cf2f0 con 0x7f69600689e0 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 -- 172.21.15.49:0/3026645103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 msgr2=0x560f96df3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 0x560f96df3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 -- 172.21.15.49:0/3026645103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 msgr2=0x560f96dfc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6964e5a640 1 -- 172.21.15.49:0/3026645103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f96dba500 con 0x560f96dfcd30 2024-03-16T08:17:35.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.401+0000 7f6965e5c640 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 0x560f96df3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f6964e5a640 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96dff120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f695c036040 tx=0x7f695c0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6928317af6f1f2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f6965e5c640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 0x7f694c0a4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6958005c40 tx=0x7f6958009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f694b7fe640 1 -- 172.21.15.49:0/3026645103 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c042070 con 0x560f96dfcd30 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f694b7fe640 1 -- 172.21.15.49:0/3026645103 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c02c8b0 con 0x560f96dfcd30 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f694b7fe640 1 -- 172.21.15.49:0/3026645103 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c004b90 con 0x560f96dfcd30 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f696a582740 1 -- 172.21.15.49:0/3026645103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 msgr2=0x560f96dff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.402+0000 7f696a582740 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96dff120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f695c036040 tx=0x7f695c0095e0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- 172.21.15.49:0/3026645103 shutdown_connections 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96dff120 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- 172.21.15.49:0/3026645103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df3830 0x560f96df3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- 172.21.15.49:0/3026645103 >> 172.21.15.49:0/3026645103 conn(0x560f96dedf60 msgr2=0x560f96dee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- 172.21.15.49:0/3026645103 shutdown_connections 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- 172.21.15.49:0/3026645103 wait complete. 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- start start 2024-03-16T08:17:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 Processor -- start 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df3830 0x560f96df9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96dfa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f96dffac0 con 0x560f96df4150 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560f96d13230 con 0x560f96df3830 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f96dffc40 con 0x560f96dfcd30 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.403+0000 7f696a582740 10 client.0 _ll_register_callbacks cb 0x560f96d4f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50626/0 (socket says 172.21.15.49:50626) 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f696565b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df3830 0x560f96df9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 -- 172.21.15.49:0/1955661797 learned_addr learned my addr 172.21.15.49:0/1955661797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6965e5c640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96dfa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 -- 172.21.15.49:0/1955661797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 msgr2=0x560f96dfa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96dfa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 -- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df3830 msgr2=0x560f96df9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df3830 0x560f96df9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 -- 172.21.15.49:0/1955661797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f96d22100 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f695c006010 tx=0x7f695c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db82f577f77ace57 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c042030 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f695c004030 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c02c8b0 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f695c056050 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f696a582740 10 client.15564 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.404+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560f96a6b3f0 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c02c8b0 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f695c003680 con 0x560f96df4150 2024-03-16T08:17:35.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 10 client.15564 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 20 client.15564 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e0cf40 0x560f96e0f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 4 client.15564 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f96d649d0 con 0x560f96e0cf40 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.405+0000 7f696565b640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e0cf40 0x560f96e0f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.406+0000 7f6948ff9640 10 client.15564 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:35.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.406+0000 7f696565b640 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e0cf40 0x560f96e0f330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54ca4c9307aab910 server_cookie=20fbb52f567a11b7 in_seq=0 out_seq=0 2024-03-16T08:17:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.406+0000 7f6948ff9640 1 -- 172.21.15.49:0/1955661797 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560f96d649d0 con 0x560f96e0cf40 2024-03-16T08:17:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.406+0000 7f6948ff9640 10 client.15564 handle_command_reply: tid=0 2024-03-16T08:17:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 client.15564 shutdown 2024-03-16T08:17:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 msgr2=0x560f96dfa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f695c006010 tx=0x7f695c02c800 comp rx=0 tx=0).stop 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 shutdown_connections 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e0cf40 0x560f96e0f330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96dfa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df4150 0x560f96dfa300 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 --2- 172.21.15.49:0/1955661797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df3830 0x560f96df9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 >> 172.21.15.49:0/1955661797 conn(0x560f96dedf60 msgr2=0x560f96defcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f6948ff9640 0 client.15564 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 shutdown_connections 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 1 -- 172.21.15.49:0/1955661797 wait complete. 2024-03-16T08:17:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.407+0000 7f696a582740 20 client.15564 trim_cache size 0 max 16384 2024-03-16T08:17:35.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.420+0000 7f696a582740 1 Processor -- start 2024-03-16T08:17:35.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.420+0000 7f696a582740 1 -- start start 2024-03-16T08:17:35.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df4150 0x560f96e08860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96defac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69601b1d80 con 0x560f96df3830 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69601af650 con 0x560f96df4150 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696a582740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ad480 con 0x560f96dfcd30 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50642/0 (socket says 172.21.15.49:50642) 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 -- 172.21.15.49:0/948043520 learned_addr learned my addr 172.21.15.49:0/948043520 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f6965e5c640 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96defac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f6964e5a640 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df4150 0x560f96e08860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 -- 172.21.15.49:0/948043520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 msgr2=0x560f96defac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96defac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 -- 172.21.15.49:0/948043520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df4150 msgr2=0x560f96e08860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df4150 0x560f96e08860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 -- 172.21.15.49:0/948043520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f96b357d0 con 0x560f96df3830 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f696565b640 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6950097c90 tx=0x7f69500d1780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d51818dcedcd56d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f694affd640 1 -- 172.21.15.49:0/948043520 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695009b190 con 0x560f96df3830 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f694affd640 1 -- 172.21.15.49:0/948043520 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695009bf30 con 0x560f96df3830 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.421+0000 7f694affd640 1 -- 172.21.15.49:0/948043520 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950097ea0 con 0x560f96df3830 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- 172.21.15.49:0/948043520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 msgr2=0x560f96dfc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6950097c90 tx=0x7f69500d1780 comp rx=0 tx=0).stop 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- 172.21.15.49:0/948043520 shutdown_connections 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96dfcd30 0x560f96defac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96df4150 0x560f96e08860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 --2- 172.21.15.49:0/948043520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96dfc480 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- 172.21.15.49:0/948043520 >> 172.21.15.49:0/948043520 conn(0x560f96dedf60 msgr2=0x560f96def3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- 172.21.15.49:0/948043520 shutdown_connections 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- 172.21.15.49:0/948043520 wait complete. 2024-03-16T08:17:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 -- start start 2024-03-16T08:17:35.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.422+0000 7f696a582740 1 Processor -- start 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 0x560f96df6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96df6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69601adb10 con 0x560f96df3830 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69601aef80 con 0x560f96dfcd30 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ae1e0 con 0x560f96df4150 2024-03-16T08:17:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696a582740 10 client.0 _ll_register_callbacks cb 0x7f69500cc860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50654/0 (socket says 172.21.15.49:50654) 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f6964e5a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 0x560f96df6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 -- 172.21.15.49:0/1302420529 learned_addr learned my addr 172.21.15.49:0/1302420529 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f6965e5c640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96df6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 -- 172.21.15.49:0/1302420529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 msgr2=0x560f96df6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 0x560f96df6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 -- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 msgr2=0x560f96df6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96df6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 -- 172.21.15.49:0/1302420529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f96df7200 con 0x560f96df3830 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 0x560f96df6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f696565b640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695000e9c0 tx=0x7f69500cc930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff54f4dfc7e140cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.423+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950065060 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6950099b30 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695009bf30 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f69500b8030 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 10 client.15570 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560f96e72e80 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695009bf30 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f69500a7080 con 0x560f96df3830 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 10 client.15570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 20 client.15570 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e73550 0x560f96e75940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 4 client.15570 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f96d5ed20 con 0x560f96e73550 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.424+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e73550 0x560f96e75940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f6948ff9640 10 client.15570 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f6964e5a640 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e73550 0x560f96e75940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7b21a969d1a72cd server_cookie=7154f1f83a61962e in_seq=0 out_seq=0 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f6948ff9640 1 -- 172.21.15.49:0/1302420529 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f96d5ed20 con 0x560f96e73550 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f6948ff9640 10 client.15570 handle_command_reply: tid=0 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 client.15570 shutdown 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 msgr2=0x560f96e05080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695000e9c0 tx=0x7f69500cc930 comp rx=0 tx=0).stop 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 shutdown_connections 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x560f96e73550 0x560f96e75940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560f96dfcd30 0x560f96df6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f96df4150 0x560f96df6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f6948ff9640 0 client.15570 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 --2- 172.21.15.49:0/1302420529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f96df3830 0x560f96e05080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 >> 172.21.15.49:0/1302420529 conn(0x560f96dedf60 msgr2=0x560f96defc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 shutdown_connections 2024-03-16T08:17:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 1 -- 172.21.15.49:0/1302420529 wait complete. 2024-03-16T08:17:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.425+0000 7f696a582740 20 client.15570 trim_cache size 0 max 16384 2024-03-16T08:17:35.433 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 msgr2=0x7f694c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 0x7f694c0a4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6958005c40 tx=0x7f6958009500 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 msgr2=0x7f69601006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f695000e9f0 tx=0x7f695000edf0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 shutdown_connections 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f694c0a2a30 0x7f694c0a4ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6960107830 0x7f6960101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69600693a0 0x7f6960100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 --2- 172.21.15.49:0/3176959936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69600689e0 0x7f69601006a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 >> 172.21.15.49:0/3176959936 conn(0x7f69600777e0 msgr2=0x7f69600fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 shutdown_connections 2024-03-16T08:17:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.426+0000 7f69678bf640 1 -- 172.21.15.49:0/3176959936 wait complete. 2024-03-16T08:17:35.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:17:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 -- 172.21.15.49:0/2905629583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c0693a0 msgr2=0x7fe61c10e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 --2- 172.21.15.49:0/2905629583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c0693a0 0x7fe61c10e640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe614009cd0 tx=0x7fe61402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 -- 172.21.15.49:0/2905629583 shutdown_connections 2024-03-16T08:17:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 --2- 172.21.15.49:0/2905629583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c0693a0 0x7fe61c10e640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 --2- 172.21.15.49:0/2905629583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0689e0 0x7fe61c068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 --2- 172.21.15.49:0/2905629583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c107830 0x7fe61c107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 -- 172.21.15.49:0/2905629583 >> 172.21.15.49:0/2905629583 conn(0x7fe61c0777e0 msgr2=0x7fe61c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 -- 172.21.15.49:0/2905629583 shutdown_connections 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.548+0000 7fe623487640 1 -- 172.21.15.49:0/2905629583 wait complete. 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 Processor -- start 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 -- start start 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 0x7fe61c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0693a0 0x7fe61c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c111b40 con 0x7fe61c107830 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1119c0 con 0x7fe61c0689e0 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe623487640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c111cc0 con 0x7fe61c0693a0 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50672/0 (socket says 172.21.15.49:50672) 2024-03-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 -- 172.21.15.49:0/1998183067 learned_addr learned my addr 172.21.15.49:0/1998183067 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621223640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 0x7fe61c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe620a22640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0693a0 0x7fe61c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 -- 172.21.15.49:0/1998183067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0693a0 msgr2=0x7fe61c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0693a0 0x7fe61c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 -- 172.21.15.49:0/1998183067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 msgr2=0x7fe61c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 0x7fe61c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621a24640 1 -- 172.21.15.49:0/1998183067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c19d050 con 0x7fe61c107830 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.549+0000 7fe621223640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 0x7fe61c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe621a24640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe614004af0 tx=0x7fe614032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe6127fc640 1 -- 172.21.15.49:0/1998183067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614042020 con 0x7fe61c107830 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c19d2e0 con 0x7fe61c107830 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe6127fc640 1 -- 172.21.15.49:0/1998183067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe614032e80 con 0x7fe61c107830 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a4bd0 con 0x7fe61c107830 2024-03-16T08:17:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.550+0000 7fe6127fc640 1 -- 172.21.15.49:0/1998183067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614039040 con 0x7fe61c107830 2024-03-16T08:17:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 Processor -- start 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe6127fc640 1 -- 172.21.15.49:0/1998183067 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fe614045020 con 0x7fe61c107830 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 -- start start 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc96fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc97b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5613fc8d3490 con 0x5613fc970150 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5613fc87bc90 con 0x5613fc978d30 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe62606a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5613fc8cf530 con 0x5613fc96f830 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe621223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe621223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50682/0 (socket says 172.21.15.49:50682) 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe620a22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc96fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe621223640 1 -- 172.21.15.49:0/184067253 learned_addr learned my addr 172.21.15.49:0/184067253 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.553+0000 7fe620a22640 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc97b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe6127fc640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 0x7fe6080a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 -- 172.21.15.49:0/184067253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 msgr2=0x5613fc96fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc96fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 -- 172.21.15.49:0/184067253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 msgr2=0x5613fc97b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc97b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 -- 172.21.15.49:0/184067253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613fc936500 con 0x5613fc970150 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe620a22640 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc96fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621a24640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 0x7fe6080a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe6127fc640 1 -- 172.21.15.49:0/1998183067 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe6140f0a60 con 0x7fe61c107830 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe621223640 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe6180114f0 tx=0x7fe6180119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305f33c533037adc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe6077fe640 1 -- 172.21.15.49:0/184067253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61801e070 con 0x5613fc970150 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe6077fe640 1 -- 172.21.15.49:0/184067253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe618011250 con 0x5613fc970150 2024-03-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.554+0000 7fe6077fe640 1 -- 172.21.15.49:0/184067253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618019990 con 0x5613fc970150 2024-03-16T08:17:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- 172.21.15.49:0/184067253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 msgr2=0x5613fc9787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe6180114f0 tx=0x7fe6180119b0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe621a24640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 0x7fe6080a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ee270 tx=0x7fe6140dd040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- 172.21.15.49:0/184067253 shutdown_connections 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc97b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc9787f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 --2- 172.21.15.49:0/184067253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc96fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- 172.21.15.49:0/184067253 >> 172.21.15.49:0/184067253 conn(0x5613fc969f60 msgr2=0x5613fc96a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- 172.21.15.49:0/184067253 shutdown_connections 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- 172.21.15.49:0/184067253 wait complete. 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.555+0000 7fe62606a740 1 -- start start 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 Processor -- start 2024-03-16T08:17:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc975cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc976770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5613fc97bb20 con 0x5613fc970150 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5613fc88f230 con 0x5613fc978d30 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5613fc97bca0 con 0x5613fc96f830 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe62606a740 10 client.0 _ll_register_callbacks cb 0x5613fc8cb010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50684/0 (socket says 172.21.15.49:50684) 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 -- 172.21.15.49:0/1020028464 learned_addr learned my addr 172.21.15.49:0/1020028464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe620a22640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc975cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe620a22640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc976770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 -- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 msgr2=0x5613fc975cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc975cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 -- 172.21.15.49:0/1020028464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 msgr2=0x5613fc976770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc976770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.556+0000 7fe621223640 1 -- 172.21.15.49:0/1020028464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613fc89e100 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe621223640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe618019cc0 tx=0x7fe6180194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bd9562984230f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618029060 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe618005050 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe618011250 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe61801e040 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 10 client.15582 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613fc5e73f0 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe618011250 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe618004c90 con 0x5613fc970150 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 20 client.15582 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc988ff0 0x5613fc98b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 4 client.15582 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.557+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613fc8e09d0 con 0x5613fc988ff0 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.558+0000 7fe620a22640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc988ff0 0x5613fc98b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.558+0000 7fe620a22640 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc988ff0 0x5613fc98b3e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c2b731b6076b6ff server_cookie=8b9a3d46c7f3249e in_seq=0 out_seq=0 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.558+0000 7fe604ff9640 10 client.15582 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe604ff9640 1 -- 172.21.15.49:0/1020028464 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613fc8e09d0 con 0x5613fc988ff0 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe604ff9640 10 client.15582 handle_command_reply: tid=0 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 client.15582 shutdown 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 msgr2=0x5613fc976230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe618019cc0 tx=0x7fe6180194d0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 shutdown_connections 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc988ff0 0x5613fc98b3e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc976770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc970150 0x5613fc976230 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe604ff9640 0 client.15582 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 --2- 172.21.15.49:0/1020028464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc96f830 0x5613fc975cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 >> 172.21.15.49:0/1020028464 conn(0x5613fc969f60 msgr2=0x5613fc96bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 shutdown_connections 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 1 -- 172.21.15.49:0/1020028464 wait complete. 2024-03-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.559+0000 7fe62606a740 20 client.15582 trim_cache size 0 max 16384 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.572+0000 7fe62606a740 1 Processor -- start 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.572+0000 7fe62606a740 1 -- start start 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 0x5613fc984920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc990b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1afda0 con 0x5613fc96f830 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1ad670 con 0x5613fc978d30 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe62606a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1ab4a0 con 0x5613fc970150 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50690/0 (socket says 172.21.15.49:50690) 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 -- 172.21.15.49:0/3262856450 learned_addr learned my addr 172.21.15.49:0/3262856450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe621223640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 0x5613fc984920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc990b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 -- 172.21.15.49:0/3262856450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 msgr2=0x5613fc984920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 0x5613fc984920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 -- 172.21.15.49:0/3262856450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 msgr2=0x5613fc990b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc990b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 -- 172.21.15.49:0/3262856450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613fc6b17d0 con 0x5613fc96f830 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe621223640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 0x5613fc984920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe620a22640 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009730 tx=0x7fe60c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acac938e398af63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe606ffd640 1 -- 172.21.15.49:0/3262856450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c012da0 con 0x5613fc96f830 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe606ffd640 1 -- 172.21.15.49:0/3262856450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c0285a0 con 0x5613fc96f830 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.573+0000 7fe606ffd640 1 -- 172.21.15.49:0/3262856450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00e730 con 0x5613fc96f830 2024-03-16T08:17:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 -- 172.21.15.49:0/3262856450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 msgr2=0x5613fc978420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009730 tx=0x7fe60c005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 -- 172.21.15.49:0/3262856450 shutdown_connections 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc978d30 0x5613fc990b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc970150 0x5613fc984920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 --2- 172.21.15.49:0/3262856450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc978420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 -- 172.21.15.49:0/3262856450 >> 172.21.15.49:0/3262856450 conn(0x5613fc969f60 msgr2=0x5613fc976540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 -- 172.21.15.49:0/3262856450 shutdown_connections 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.574+0000 7fe62606a740 1 -- 172.21.15.49:0/3262856450 wait complete. 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 -- start start 2024-03-16T08:17:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 Processor -- start 2024-03-16T08:17:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 0x5613fc973170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc978d30 0x5613fc9736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1abb30 con 0x5613fc96f830 2024-03-16T08:17:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1acfa0 con 0x5613fc970150 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1ac200 con 0x5613fc978d30 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe62606a740 10 client.0 _ll_register_callbacks cb 0x7fe61801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe620a22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe621223640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 0x5613fc973170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe620a22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50692/0 (socket says 172.21.15.49:50692) 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.575+0000 7fe620a22640 1 -- 172.21.15.49:0/3460251737 learned_addr learned my addr 172.21.15.49:0/3460251737 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe620a22640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc978d30 0x5613fc9736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 -- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc978d30 msgr2=0x5613fc9736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc978d30 0x5613fc9736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 -- 172.21.15.49:0/3460251737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 msgr2=0x5613fc972c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 -- 172.21.15.49:0/3460251737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613fc9f2880 con 0x5613fc970150 2024-03-16T08:17:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe620a22640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe621223640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 0x5613fc973170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe618019c90 tx=0x7fe61800f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568a33d33c799501 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe604ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618029040 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe618024670 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60c0285a0 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe61803d050 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe62606a740 10 client.25475 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.576+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613fc9f3b80 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c0285a0 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe618036060 con 0x5613fc970150 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 10 client.25475 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 10 client.25475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 20 client.25475 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc9f4260 0x5613fc9f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 4 client.25475 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613fc8dad20 con 0x5613fc9f4260 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe620a22640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc9f4260 0x5613fc9f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe620a22640 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc9f4260 0x5613fc9f6680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51998702a678a885 server_cookie=304dacb8b5d70bad in_seq=0 out_seq=0 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.577+0000 7fe604ff9640 10 client.25475 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe604ff9640 1 -- 172.21.15.49:0/3460251737 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613fc8dad20 con 0x5613fc9f4260 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe604ff9640 10 client.25475 handle_command_reply: tid=0 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 client.25475 shutdown 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 msgr2=0x5613fc973170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 0x5613fc973170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe618019c90 tx=0x7fe61800f800 comp rx=0 tx=0).stop 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 shutdown_connections 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5613fc9f4260 0x5613fc9f6680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5613fc978d30 0x5613fc9736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5613fc970150 0x5613fc973170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe604ff9640 0 client.25475 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 --2- 172.21.15.49:0/3460251737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613fc96f830 0x5613fc972c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 >> 172.21.15.49:0/3460251737 conn(0x5613fc969f60 msgr2=0x5613fc975c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 shutdown_connections 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 1 -- 172.21.15.49:0/3460251737 wait complete. 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe62606a740 20 client.25475 trim_cache size 0 max 16384 2024-03-16T08:17:35.586 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 msgr2=0x7fe6080a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 0x7fe6080a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ee270 tx=0x7fe6140dd040 comp rx=0 tx=0).stop 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 msgr2=0x7fe61c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe614004af0 tx=0x7fe614032c90 comp rx=0 tx=0).stop 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 shutdown_connections 2024-03-16T08:17:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe6080a2a00 0x7fe6080a4ec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61c107830 0x7fe61c19c920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0693a0 0x7fe61c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 --2- 172.21.15.49:0/1998183067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe61c0689e0 0x7fe61c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.578+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 >> 172.21.15.49:0/1998183067 conn(0x7fe61c0777e0 msgr2=0x7fe61c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.579+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 shutdown_connections 2024-03-16T08:17:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.579+0000 7fe623487640 1 -- 172.21.15.49:0/1998183067 wait complete. 2024-03-16T08:17:35.590 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:17:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- 172.21.15.49:0/1256609257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674102120 msgr2=0x7f667410e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 --2- 172.21.15.49:0/1256609257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674102120 0x7f667410e630 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f666c009cd0 tx=0x7f666c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- 172.21.15.49:0/1256609257 shutdown_connections 2024-03-16T08:17:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 --2- 172.21.15.49:0/1256609257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674102120 0x7f667410e630 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 --2- 172.21.15.49:0/1256609257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674101760 0x7f6674101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 --2- 172.21.15.49:0/1256609257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674107770 0x7f6674107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- 172.21.15.49:0/1256609257 >> 172.21.15.49:0/1256609257 conn(0x7f66740fd270 msgr2=0x7f66740ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- 172.21.15.49:0/1256609257 shutdown_connections 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- 172.21.15.49:0/1256609257 wait complete. 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 Processor -- start 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.700+0000 7f667c485640 1 -- start start 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674102120 0x7f66741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674107770 0x7f667419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111b80 con 0x7f6674101760 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111a00 con 0x7f6674102120 2024-03-16T08:17:35.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111d00 con 0x7f6674107770 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50708/0 (socket says 172.21.15.49:50708) 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 -- 172.21.15.49:0/2142191542 learned_addr learned my addr 172.21.15.49:0/2142191542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f6679a20640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674102120 0x7f66741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667aa22640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674107770 0x7f667419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 -- 172.21.15.49:0/2142191542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674107770 msgr2=0x7f667419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674107770 0x7f667419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 -- 172.21.15.49:0/2142191542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674102120 msgr2=0x7f66741985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674102120 0x7f66741985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 -- 172.21.15.49:0/2142191542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667419d050 con 0x7f6674101760 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667a221640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f667000ea60 tx=0x7f667000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f666b7fe640 1 -- 172.21.15.49:0/2142191542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667000cde0 con 0x7f6674101760 2024-03-16T08:17:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667419d340 con 0x7f6674101760 2024-03-16T08:17:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f666b7fe640 1 -- 172.21.15.49:0/2142191542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66700035f0 con 0x7f6674101760 2024-03-16T08:17:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f666b7fe640 1 -- 172.21.15.49:0/2142191542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670010670 con 0x7f6674101760 2024-03-16T08:17:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.701+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a4bd0 con 0x7f6674101760 2024-03-16T08:17:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.703+0000 7f666b7fe640 1 -- 172.21.15.49:0/2142191542 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f66700037a0 con 0x7f6674101760 2024-03-16T08:17:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.704+0000 7f667f079740 1 Processor -- start 2024-03-16T08:17:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.704+0000 7f667f079740 1 -- start start 2024-03-16T08:17:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65ab4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab5150 0x564e65abd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e65a18490 con 0x564e65abdd30 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564e659c0c90 con 0x564e65ab4830 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667f079740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564e65a14530 con 0x564e65ab5150 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f666b7fe640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 0x7f66600a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667a221640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab5150 0x564e65abd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50718/0 (socket says 172.21.15.49:50718) 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 -- 172.21.15.49:0/1839601030 learned_addr learned my addr 172.21.15.49:0/1839601030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667aa22640 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65ab4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667aa22640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 0x7f66600a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 -- 172.21.15.49:0/1839601030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab5150 msgr2=0x564e65abd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab5150 0x564e65abd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f666b7fe640 1 -- 172.21.15.49:0/2142191542 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f66700cf270 con 0x7f6674101760 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 -- 172.21.15.49:0/1839601030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 msgr2=0x564e65ab4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65ab4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 -- 172.21.15.49:0/1839601030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e65a7b500 con 0x564e65abdd30 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f667aa22640 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65ab4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6679a20640 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6664009c90 tx=0x7f6664011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe585bacbc91414 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.705+0000 7f6668bfe640 1 -- 172.21.15.49:0/1839601030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401e070 con 0x564e65abdd30 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667aa22640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 0x7f66600a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f666c005cc0 tx=0x7f666c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f6668bfe640 1 -- 172.21.15.49:0/1839601030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664019b10 con 0x564e65abdd30 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f6668bfe640 1 -- 172.21.15.49:0/1839601030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666400ea70 con 0x564e65abdd30 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 -- 172.21.15.49:0/1839601030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 msgr2=0x564e65ac0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6664009c90 tx=0x7f6664011810 comp rx=0 tx=0).stop 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 -- 172.21.15.49:0/1839601030 shutdown_connections 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65abdd30 0x564e65ac0120 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab5150 0x564e65abd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 --2- 172.21.15.49:0/1839601030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65ab4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 -- 172.21.15.49:0/1839601030 >> 172.21.15.49:0/1839601030 conn(0x564e65aaef60 msgr2=0x564e65aaf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 -- 172.21.15.49:0/1839601030 shutdown_connections 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.706+0000 7f667f079740 1 -- 172.21.15.49:0/1839601030 wait complete. 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 -- start start 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 Processor -- start 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 0x564e65abb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65abb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e65ac0b20 con 0x564e65ab4830 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564e659d4230 con 0x564e65ab5150 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564e65ac0ca0 con 0x564e65abdd30 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667f079740 10 client.0 _ll_register_callbacks cb 0x564e65a10010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667aa22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65abb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 0x564e65abb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50722/0 (socket says 172.21.15.49:50722) 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667aa22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65abb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:51742/0 (socket says 172.21.15.49:51742) 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667a221640 1 -- 172.21.15.49:0/1394544006 learned_addr learned my addr 172.21.15.49:0/1394544006 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 -- 172.21.15.49:0/1394544006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 msgr2=0x564e65abb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65abb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 -- 172.21.15.49:0/1394544006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 msgr2=0x564e65abace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f6679a20640 1 -- 172.21.15.49:0/1394544006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e659e3100 con 0x564e65ab5150 2024-03-16T08:17:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.707+0000 7f667a221640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f6679a20640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 0x564e65abb220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f666401c040 tx=0x7f6664009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54250e4ec909ecbf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f665dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401e050 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6664005450 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6664019b10 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6664005650 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f667f079740 10 client.25487 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e6572c3f0 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.708+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664019b10 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f666400ecd0 con 0x564e65ab5150 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 10 client.25487 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 20 client.25487 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65acdf20 0x564e65ad0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 4 client.25487 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e65a259d0 con 0x564e65acdf20 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667a221640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65acdf20 0x564e65ad0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f667a221640 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65acdf20 0x564e65ad0310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5502bebd7a677913 server_cookie=d7f68358927baf18 in_seq=0 out_seq=0 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.709+0000 7f665dffb640 10 client.25487 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f665dffb640 1 -- 172.21.15.49:0/1394544006 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564e65a259d0 con 0x564e65acdf20 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f665dffb640 10 client.25487 handle_command_reply: tid=0 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 client.25487 shutdown 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 msgr2=0x564e65abb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 0x564e65abb220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f666401c040 tx=0x7f6664009850 comp rx=0 tx=0).stop 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 shutdown_connections 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65acdf20 0x564e65ad0310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65abb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab5150 0x564e65abb220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 --2- 172.21.15.49:0/1394544006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab4830 0x564e65abace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f665dffb640 0 client.25487 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.710+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 >> 172.21.15.49:0/1394544006 conn(0x564e65aaef60 msgr2=0x564e65ab0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.711+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 shutdown_connections 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.711+0000 7f667f079740 1 -- 172.21.15.49:0/1394544006 wait complete. 2024-03-16T08:17:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.711+0000 7f667f079740 20 client.25487 trim_cache size 0 max 16384 2024-03-16T08:17:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 Processor -- start 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 -- start start 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65abd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65aade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66741afdd0 con 0x564e65ab5150 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ad6a0 con 0x564e65ab4830 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667f079740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ab4d0 con 0x564e65abdd30 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50734/0 (socket says 172.21.15.49:50734) 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 -- 172.21.15.49:0/886329064 learned_addr learned my addr 172.21.15.49:0/886329064 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667aa22640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65aade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667a221640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65abd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 -- 172.21.15.49:0/886329064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 msgr2=0x564e65aade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65aade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 -- 172.21.15.49:0/886329064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 msgr2=0x564e65abd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65abd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 -- 172.21.15.49:0/886329064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e65ab5b00 con 0x564e65ab5150 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667aa22640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65aade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f667a221640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65abd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.724+0000 7f6679a20640 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6664009c90 tx=0x7f6664009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f76c5f4bfa3b4e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f665ffff640 1 -- 172.21.15.49:0/886329064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401e040 con 0x564e65ab5150 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f665ffff640 1 -- 172.21.15.49:0/886329064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666401f040 con 0x564e65ab5150 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f665ffff640 1 -- 172.21.15.49:0/886329064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664005250 con 0x564e65ab5150 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 -- 172.21.15.49:0/886329064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 msgr2=0x564e65ac98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6664009c90 tx=0x7f6664009e40 comp rx=0 tx=0).stop 2024-03-16T08:17:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 -- 172.21.15.49:0/886329064 shutdown_connections 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65abdd30 0x564e65aade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ac98c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 --2- 172.21.15.49:0/886329064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65ab4830 0x564e65abd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 -- 172.21.15.49:0/886329064 >> 172.21.15.49:0/886329064 conn(0x564e65aaef60 msgr2=0x564e65aad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 -- 172.21.15.49:0/886329064 shutdown_connections 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.725+0000 7f667f079740 1 -- 172.21.15.49:0/886329064 wait complete. 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 -- start start 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 Processor -- start 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab4830 0x564e65ab7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65abdd30 0x564e65ab8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66741abb60 con 0x564e65ab5150 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66741acfd0 con 0x564e65abdd30 2024-03-16T08:17:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ac230 con 0x564e65ab4830 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667f079740 10 client.0 _ll_register_callbacks cb 0x7f666401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50742/0 (socket says 172.21.15.49:50742) 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 -- 172.21.15.49:0/2158122544 learned_addr learned my addr 172.21.15.49:0/2158122544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667a221640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab4830 0x564e65ab7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f667aa22640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65abdd30 0x564e65ab8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 -- 172.21.15.49:0/2158122544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab4830 msgr2=0x564e65ab7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab4830 0x564e65ab7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 -- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65abdd30 msgr2=0x564e65ab8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65abdd30 0x564e65ab8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.726+0000 7f6679a20640 1 -- 172.21.15.49:0/2158122544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e657f67d0 con 0x564e65ab5150 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f6679a20640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6664000c00 tx=0x7f6664004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9983d352d6bb38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401e070 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6664005250 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666401f040 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6664043020 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 10 client.15606 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e65b343c0 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666401f040 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6664035030 con 0x564e65ab5150 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 10 client.15606 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 20 client.15606 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65b34a70 0x564e65b36e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 4 client.15606 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.727+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e65a1fd20 con 0x564e65b34a70 2024-03-16T08:17:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667a221640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65b34a70 0x564e65b36e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667a221640 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65b34a70 0x564e65b36e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdae3fbcfefff9fb server_cookie=cb3d74d3b28f6099 in_seq=0 out_seq=0 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f665dffb640 10 client.15606 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f665dffb640 1 -- 172.21.15.49:0/2158122544 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e65a1fd20 con 0x564e65b34a70 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f665dffb640 10 client.15606 handle_command_reply: tid=0 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 client.15606 shutdown 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 msgr2=0x564e65ab7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6664000c00 tx=0x7f6664004dc0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 shutdown_connections 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x564e65b34a70 0x564e65b36e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564e65abdd30 0x564e65ab8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e65ab5150 0x564e65ab7fd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 --2- 172.21.15.49:0/2158122544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e65ab4830 0x564e65ab7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f665dffb640 0 client.15606 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 >> 172.21.15.49:0/2158122544 conn(0x564e65aaef60 msgr2=0x564e65aadfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 shutdown_connections 2024-03-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 1 -- 172.21.15.49:0/2158122544 wait complete. 2024-03-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.728+0000 7f667f079740 20 client.15606 trim_cache size 0 max 16384 2024-03-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 msgr2=0x7f66600a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 0x7f66600a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f666c005cc0 tx=0x7f666c005c50 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 msgr2=0x7f6674198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f667000ea60 tx=0x7f667000ef20 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 shutdown_connections 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66600a2a00 0x7f66600a4ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6674107770 0x7f667419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6674102120 0x7f66741985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 --2- 172.21.15.49:0/2142191542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6674101760 0x7f6674198060 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 >> 172.21.15.49:0/2142191542 conn(0x7f66740fd270 msgr2=0x7f66740fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 shutdown_connections 2024-03-16T08:17:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:35.729+0000 7f667c485640 1 -- 172.21.15.49:0/2142191542 wait complete. 2024-03-16T08:17:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:36 smithi154 ceph-mon[36471]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-16T08:17:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:36 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1072086561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:36.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:36 smithi049 ceph-mon[31942]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-16T08:17:36.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:36 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1072086561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:36.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:36 smithi129 ceph-mon[36561]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-16T08:17:36.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:36 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1072086561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:36.741 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:17:36.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:17:36.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 -- 172.21.15.49:0/3356460392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 msgr2=0x7f25d810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 --2- 172.21.15.49:0/3356460392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d810d520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009cd0 tx=0x7f25d002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 -- 172.21.15.49:0/3356460392 shutdown_connections 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 --2- 172.21.15.49:0/3356460392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d810d520 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 --2- 172.21.15.49:0/3356460392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d8100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 --2- 172.21.15.49:0/3356460392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 0x7f25d8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 -- 172.21.15.49:0/3356460392 >> 172.21.15.49:0/3356460392 conn(0x7f25d80777e0 msgr2=0x7f25d8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 -- 172.21.15.49:0/3356460392 shutdown_connections 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.865+0000 7f25df15b640 1 -- 172.21.15.49:0/3356460392 wait complete. 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 Processor -- start 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 -- start start 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 0x7f25d819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8110a70 con 0x7f25d8101010 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81108f0 con 0x7f25d8100650 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25df15b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8110bf0 con 0x7f25d8106660 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50764/0 (socket says 172.21.15.49:50764) 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25d7fff640 1 -- 172.21.15.49:0/3527865257 learned_addr learned my addr 172.21.15.49:0/3527865257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.866+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 0x7f25d819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 -- 172.21.15.49:0/3527865257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 msgr2=0x7f25d819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 0x7f25d819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 -- 172.21.15.49:0/3527865257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 msgr2=0x7f25d81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 -- 172.21.15.49:0/3527865257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d819d0b0 con 0x7f25d8101010 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f25c400ea60 tx=0x7f25c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d5ffb640 1 -- 172.21.15.49:0/3527865257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c400cde0 con 0x7f25d8101010 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d5ffb640 1 -- 172.21.15.49:0/3527865257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c40035f0 con 0x7f25d8101010 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d819d3a0 con 0x7f25d8101010 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25d5ffb640 1 -- 172.21.15.49:0/3527865257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4010670 con 0x7f25d8101010 2024-03-16T08:17:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.867+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81a4c30 con 0x7f25d8101010 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.870+0000 7f25d5ffb640 1 -- 172.21.15.49:0/3527865257 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f25c40037a0 con 0x7f25d8101010 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 Processor -- start 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 -- start start 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9bd710 0x55f51d9bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9c5da0 0x55f51d9c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f51d9200d0 con 0x55f51d9bca30 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f51d8c8c70 con 0x55f51d9c5da0 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25e1e23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f51d91c170 con 0x55f51d9bd710 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50780/0 (socket says 172.21.15.49:50780) 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 -- 172.21.15.49:0/2559262564 learned_addr learned my addr 172.21.15.49:0/2559262564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9bd710 0x55f51d9bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25d7fff640 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9c5da0 0x55f51d9c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25d5ffb640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 0x7f25c80a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 -- 172.21.15.49:0/2559262564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9bd710 msgr2=0x55f51d9bc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9bd710 0x55f51d9bc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 0x7f25c80a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 -- 172.21.15.49:0/2559262564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9c5da0 msgr2=0x55f51d9c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9c5da0 0x55f51d9c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25dcef7640 1 -- 172.21.15.49:0/2559262564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51d6fc7e0 con 0x55f51d9bca30 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.871+0000 7f25d5ffb640 1 -- 172.21.15.49:0/3527865257 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f25c40cf2a0 con 0x7f25d8101010 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25dcef7640 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0114c0 tx=0x7f25cc011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda25c56863257ec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25c2ffd640 1 -- 172.21.15.49:0/2559262564 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc004020 con 0x55f51d9bca30 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 0x7f25c80a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005cc0 tx=0x7f25d002c0f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25c2ffd640 1 -- 172.21.15.49:0/2559262564 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc0041c0 con 0x55f51d9bca30 2024-03-16T08:17:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25c2ffd640 1 -- 172.21.15.49:0/2559262564 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00f8f0 con 0x55f51d9bca30 2024-03-16T08:17:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25e1e23740 1 -- 172.21.15.49:0/2559262564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 msgr2=0x55f51d9bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.872+0000 7f25e1e23740 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0114c0 tx=0x7f25cc011980 comp rx=0 tx=0).stop 2024-03-16T08:17:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- 172.21.15.49:0/2559262564 shutdown_connections 2024-03-16T08:17:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9c5da0 0x55f51d9c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9bceb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- 172.21.15.49:0/2559262564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9bd710 0x55f51d9bc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- 172.21.15.49:0/2559262564 >> 172.21.15.49:0/2559262564 conn(0x55f51d983c50 msgr2=0x55f51d9b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- 172.21.15.49:0/2559262564 shutdown_connections 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- 172.21.15.49:0/2559262564 wait complete. 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- start start 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 Processor -- start 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f51d9c8c10 con 0x55f51d9bd710 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.873+0000 7f25e1e23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f51d8dc0d0 con 0x55f51d9bca30 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25e1e23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f51d9c8d90 con 0x55f51d9c5da0 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25e1e23740 10 client.0 _ll_register_callbacks cb 0x55f51d8a5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50782/0 (socket says 172.21.15.49:50782) 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 -- 172.21.15.49:0/1448332416 learned_addr learned my addr 172.21.15.49:0/1448332416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 -- 172.21.15.49:0/1448332416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 msgr2=0x55f51d9c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 -- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 msgr2=0x55f51d9c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 -- 172.21.15.49:0/1448332416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51d75cf70 con 0x55f51d9bd710 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25dcef7640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25d7fff640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f25c401f000 tx=0x7f25c40d28c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0686b6c6d40a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c409bee0 con 0x55f51d9bd710 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f25c4097e50 con 0x55f51d9bd710 2024-03-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25cc0041c0 con 0x55f51d9bd710 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25c409b7e0 con 0x55f51d9bd710 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25e1e23740 10 client.15618 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.874+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f51d5f4170 con 0x55f51d9bd710 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc0041c0 con 0x55f51d9bd710 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25c409ac30 con 0x55f51d9bd710 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 10 client.15618 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 20 client.15618 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51d9d5f00 0x55f51d9d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 4 client.15618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f51d92d610 con 0x55f51d9d5f00 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.875+0000 7f25dcef7640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51d9d5f00 0x55f51d9d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25dcef7640 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51d9d5f00 0x55f51d9d8350 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a999c20eb709a6 server_cookie=61eebd74f9f74f85 in_seq=0 out_seq=0 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25b7fff640 10 client.15618 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25b7fff640 1 -- 172.21.15.49:0/1448332416 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f51d92d610 con 0x55f51d9d5f00 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25b7fff640 10 client.15618 handle_command_reply: tid=0 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 client.15618 shutdown 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 msgr2=0x55f51d9c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f25c401f000 tx=0x7f25c40d28c0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 shutdown_connections 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51d9d5f00 0x55f51d9d8350 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9c3290 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25b7fff640 0 client.15618 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 --2- 172.21.15.49:0/1448332416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.876+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 >> 172.21.15.49:0/1448332416 conn(0x55f51d983c50 msgr2=0x55f51d9b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.877+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 shutdown_connections 2024-03-16T08:17:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.877+0000 7f25e1e23740 1 -- 172.21.15.49:0/1448332416 wait complete. 2024-03-16T08:17:36.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.877+0000 7f25e1e23740 20 client.15618 trim_cache size 0 max 16384 2024-03-16T08:17:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 Processor -- start 2024-03-16T08:17:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 -- start start 2024-03-16T08:17:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bd710 0x55f51d9d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81afcd0 con 0x55f51d9bca30 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81ad5a0 con 0x55f51d9bd710 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.894+0000 7f25e1e23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81ab3d0 con 0x55f51d9c5da0 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50792/0 (socket says 172.21.15.49:50792) 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 -- 172.21.15.49:0/3324205143 learned_addr learned my addr 172.21.15.49:0/3324205143 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bd710 0x55f51d9d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 -- 172.21.15.49:0/3324205143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 msgr2=0x55f51d9b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 -- 172.21.15.49:0/3324205143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bd710 msgr2=0x55f51d9d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bd710 0x55f51d9d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 -- 172.21.15.49:0/3324205143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51d9bba60 con 0x55f51d9bca30 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005f10 tx=0x7f25cc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af6a372a0f2db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25c27fc640 1 -- 172.21.15.49:0/3324205143 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc01d040 con 0x55f51d9bca30 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25c27fc640 1 -- 172.21.15.49:0/3324205143 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc003da0 con 0x55f51d9bca30 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.895+0000 7f25c27fc640 1 -- 172.21.15.49:0/3324205143 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00f5c0 con 0x55f51d9bca30 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- 172.21.15.49:0/3324205143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 msgr2=0x55f51d9c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005f10 tx=0x7f25cc005d30 comp rx=0 tx=0).stop 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- 172.21.15.49:0/3324205143 shutdown_connections 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51d9b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bd710 0x55f51d9d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3324205143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bca30 0x55f51d9c5410 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- 172.21.15.49:0/3324205143 >> 172.21.15.49:0/3324205143 conn(0x55f51d983c50 msgr2=0x55f51d9b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- 172.21.15.49:0/3324205143 shutdown_connections 2024-03-16T08:17:36.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- 172.21.15.49:0/3324205143 wait complete. 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 -- start start 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.896+0000 7f25e1e23740 1 Processor -- start 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51da3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81aba60 con 0x55f51d9bd710 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81aced0 con 0x55f51d9bca30 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81ac130 con 0x55f51d9c5da0 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25e1e23740 10 client.0 _ll_register_callbacks cb 0x7f25c400e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25dcef7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50806/0 (socket says 172.21.15.49:50806) 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 -- 172.21.15.49:0/3881024707 learned_addr learned my addr 172.21.15.49:0/3881024707 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25dd6f8640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51da3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 -- 172.21.15.49:0/3881024707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 msgr2=0x55f51da3b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51da3b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 -- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 msgr2=0x55f51d9ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 -- 172.21.15.49:0/3881024707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51da3b7c0 con 0x55f51d9bd710 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25d7fff640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c760 tx=0x7f25c400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e392c76be49891e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.897+0000 7f25b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40b8070 con 0x55f51d9bd710 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f25c4097e50 con 0x55f51d9bd710 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25cc003da0 con 0x55f51d9bd710 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25c407e020 con 0x55f51d9bd710 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 10 client.15624 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f51da3cb00 con 0x55f51d9bd710 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc003da0 con 0x55f51d9bd710 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25c40afbc0 con 0x55f51d9bd710 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 10 client.15624 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 20 client.15624 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51da3d270 0x55f51da3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 4 client.15624 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f51d927960 con 0x55f51da3d270 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.898+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51da3d270 0x55f51da3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.899+0000 7f25b7fff640 10 client.15624 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.899+0000 7f25dcef7640 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51da3d270 0x55f51da3f690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1c213091212bcb server_cookie=f516f9f01a8ddce9 in_seq=0 out_seq=0 2024-03-16T08:17:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25b7fff640 1 -- 172.21.15.49:0/3881024707 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f51d927960 con 0x55f51da3d270 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25b7fff640 10 client.15624 handle_command_reply: tid=0 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 client.15624 shutdown 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 msgr2=0x55f51d9ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c760 tx=0x7f25c400ef90 comp rx=0 tx=0).stop 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 shutdown_connections 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f51da3d270 0x55f51da3f690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25b7fff640 0 client.15624 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f51d9c5da0 0x55f51da3b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f51d9bd710 0x55f51d9ca5f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 --2- 172.21.15.49:0/3881024707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f51d9bca30 0x55f51d9ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 >> 172.21.15.49:0/3881024707 conn(0x55f51d983c50 msgr2=0x55f51d9b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 shutdown_connections 2024-03-16T08:17:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 1 -- 172.21.15.49:0/3881024707 wait complete. 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.900+0000 7f25e1e23740 20 client.15624 trim_cache size 0 max 16384 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "de740375-51b6-40ef-a2f9-bd0ce656a24b", 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:17:36.908 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 msgr2=0x7f25c80a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 0x7f25c80a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005cc0 tx=0x7f25d002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 msgr2=0x7f25d8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f25c400ea60 tx=0x7f25c400ef20 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 shutdown_connections 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25c80a2a00 0x7f25c80a4ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25d8106660 0x7f25d819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25d8101010 0x7f25d8198600 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 --2- 172.21.15.49:0/3527865257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d8100650 0x7f25d81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 >> 172.21.15.49:0/3527865257 conn(0x7f25d80777e0 msgr2=0x7f25d8068ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 shutdown_connections 2024-03-16T08:17:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:36.901+0000 7f25df15b640 1 -- 172.21.15.49:0/3527865257 wait complete. 2024-03-16T08:17:36.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:17:36.913 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:17:37.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/801232702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04102110 msgr2=0x7f7b0410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/801232702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04102110 0x7f7b0410e620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009cd0 tx=0x7f7afc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/801232702 shutdown_connections 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/801232702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04102110 0x7f7b0410e620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/801232702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04101750 0x7f7b04101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/801232702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 0x7f7b04107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/801232702 >> 172.21.15.49:0/801232702 conn(0x7f7b040fd240 msgr2=0x7f7b040ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/801232702 shutdown_connections 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.022+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/801232702 wait complete. 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 Processor -- start 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 -- start start 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04102110 0x7f7b04198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 0x7f7b0419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111b30 con 0x7f7b04101750 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041119b0 con 0x7f7b04102110 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0b6ae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111cb0 con 0x7f7b04107760 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50832/0 (socket says 172.21.15.49:50832) 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 -- 172.21.15.49:0/2940833973 learned_addr learned my addr 172.21.15.49:0/2940833973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b08c49640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04102110 0x7f7b04198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 0x7f7b0419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 -- 172.21.15.49:0/2940833973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 msgr2=0x7f7b0419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 0x7f7b0419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 -- 172.21.15.49:0/2940833973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04102110 msgr2=0x7f7b04198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.023+0000 7f7b0944a640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04102110 0x7f7b04198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7b0944a640 1 -- 172.21.15.49:0/2940833973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419d040 con 0x7f7b04101750 2024-03-16T08:17:37.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7b0944a640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000d950 tx=0x7f7b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7afa7fc640 1 -- 172.21.15.49:0/2940833973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00017030 con 0x7f7b04101750 2024-03-16T08:17:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7afa7fc640 1 -- 172.21.15.49:0/2940833973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b00004020 con 0x7f7b04101750 2024-03-16T08:17:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419d330 con 0x7f7b04101750 2024-03-16T08:17:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7afa7fc640 1 -- 172.21.15.49:0/2940833973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000047a0 con 0x7f7b04101750 2024-03-16T08:17:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.025+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a4bc0 con 0x7f7b04101750 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7afa7fc640 1 -- 172.21.15.49:0/2940833973 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f7b000041d0 con 0x7f7b04101750 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 Processor -- start 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 -- start start 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7afa7fc640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 0x7f7af00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 0x562ac7d85520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 0x7f7af00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7afa7fc640 1 -- 172.21.15.49:0/2940833973 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7b000ce820 con 0x7f7b04101750 2024-03-16T08:17:37.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ac7cdc500 con 0x562ac7d77e80 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ac7c84be0 con 0x562ac7d787a0 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0e370740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ac7cd85a0 con 0x562ac7d85a60 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50840/0 (socket says 172.21.15.49:50840) 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 -- 172.21.15.49:0/3685064739 learned_addr learned my addr 172.21.15.49:0/3685064739 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b08c49640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 0x562ac7d85520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 -- 172.21.15.49:0/3685064739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 msgr2=0x562ac7d87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 -- 172.21.15.49:0/3685064739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 msgr2=0x562ac7d85520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 0x562ac7d85520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b0944a640 1 -- 172.21.15.49:0/3685064739 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac7c984b0 con 0x562ac7d77e80 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b08c49640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 0x562ac7d85520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:17:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.027+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 0x7f7af00a4ec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005cc0 tx=0x7f7afc032340 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0944a640 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b000971c0 tx=0x7f7b0009ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c355a6e94b7fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7aef7fe640 1 -- 172.21.15.49:0/3685064739 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0009ad60 con 0x562ac7d77e80 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7aef7fe640 1 -- 172.21.15.49:0/3685064739 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0009eeb0 con 0x562ac7d77e80 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7aef7fe640 1 -- 172.21.15.49:0/3685064739 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0009a1b0 con 0x562ac7d77e80 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 -- 172.21.15.49:0/3685064739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 msgr2=0x562ac7d78260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b000971c0 tx=0x7f7b0009ac60 comp rx=0 tx=0).stop 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 -- 172.21.15.49:0/3685064739 shutdown_connections 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d787a0 0x562ac7d85520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 --2- 172.21.15.49:0/3685064739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d77e80 0x562ac7d78260 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.028+0000 7f7b0e370740 1 -- 172.21.15.49:0/3685064739 >> 172.21.15.49:0/3685064739 conn(0x562ac7d72880 msgr2=0x562ac7d72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- 172.21.15.49:0/3685064739 shutdown_connections 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- 172.21.15.49:0/3685064739 wait complete. 2024-03-16T08:17:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- start start 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 Processor -- start 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 0x562ac7d82ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ac7d88820 con 0x562ac7d787a0 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ac7c98160 con 0x562ac7d77e80 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ac7d889a0 con 0x562ac7d85a60 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0e370740 10 client.0 _ll_register_callbacks cb 0x562ac7c7c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b0944a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 0x562ac7d82ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50854/0 (socket says 172.21.15.49:50854) 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b08c49640 1 -- 172.21.15.49:0/712891293 learned_addr learned my addr 172.21.15.49:0/712891293 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.029+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 -- 172.21.15.49:0/712891293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 msgr2=0x562ac7d83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 -- 172.21.15.49:0/712891293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 msgr2=0x562ac7d83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 -- 172.21.15.49:0/712891293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac7aba7d0 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b08c49640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0944a640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 0x562ac7d82ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x562ac7d88450 tx=0x7f7b0009b590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3eb73481e4a78d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7aecff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000b7030 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0009eeb0 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7b00099060 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7b000a3500 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0e370740 10 client.25523 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ac79f03f0 con 0x562ac7d77e80 2024-03-16T08:17:37.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.030+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0009eeb0 con 0x562ac7d77e80 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7b00096ab0 con 0x562ac7d77e80 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 10 client.25523 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 20 client.25523 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7d95880 0x562ac7d97c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 4 client.25523 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ac7ce9a40 con 0x562ac7d95880 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b08c49640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7d95880 0x562ac7d97c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7aecff9640 10 client.25523 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.032+0000 7f7b08c49640 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7d95880 0x562ac7d97c70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee14b77febe75159 server_cookie=a81a405d86892d26 in_seq=0 out_seq=0 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7aecff9640 1 -- 172.21.15.49:0/712891293 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ac7ce9a40 con 0x562ac7d95880 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7aecff9640 10 client.25523 handle_command_reply: tid=0 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 client.25523 shutdown 2024-03-16T08:17:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 msgr2=0x562ac7d82ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 0x562ac7d82ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x562ac7d88450 tx=0x7f7b0009b590 comp rx=0 tx=0).stop 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 shutdown_connections 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7d95880 0x562ac7d97c70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d85a60 0x562ac7d83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 --2- 172.21.15.49:0/712891293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d77e80 0x562ac7d82ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7aecff9640 0 client.25523 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 >> 172.21.15.49:0/712891293 conn(0x562ac7d72880 msgr2=0x562ac7d745d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 shutdown_connections 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.033+0000 7f7b0e370740 1 -- 172.21.15.49:0/712891293 wait complete. 2024-03-16T08:17:37.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.034+0000 7f7b0e370740 20 client.25523 trim_cache size 0 max 16384 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 Processor -- start 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 -- start start 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041adb20 con 0x562ac7d787a0 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ab3f0 con 0x562ac7d85a60 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0e370740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041a9220 con 0x562ac7d77e80 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50858/0 (socket says 172.21.15.49:50858) 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b08c49640 1 -- 172.21.15.49:0/2822716183 learned_addr learned my addr 172.21.15.49:0/2822716183 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b0944a640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.051+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 -- 172.21.15.49:0/2822716183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 msgr2=0x562ac7d81a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d81a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 -- 172.21.15.49:0/2822716183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 msgr2=0x562ac7d91600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d91600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 -- 172.21.15.49:0/2822716183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac7d7d830 con 0x562ac7d787a0 2024-03-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b0944a640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d81a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7b08c49640 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7af4004310 tx=0x7f7af4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7337fc3b74431a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7aeeffd640 1 -- 172.21.15.49:0/2822716183 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40044c0 con 0x562ac7d787a0 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7aeeffd640 1 -- 172.21.15.49:0/2822716183 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af4025070 con 0x562ac7d787a0 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.052+0000 7f7aeeffd640 1 -- 172.21.15.49:0/2822716183 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003d70 con 0x562ac7d787a0 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- 172.21.15.49:0/2822716183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 msgr2=0x562ac7d7dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7af4004310 tx=0x7f7af4005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- 172.21.15.49:0/2822716183 shutdown_connections 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d7dd40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- 172.21.15.49:0/2822716183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d81a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- 172.21.15.49:0/2822716183 >> 172.21.15.49:0/2822716183 conn(0x562ac7d72880 msgr2=0x562ac7d71900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- 172.21.15.49:0/2822716183 shutdown_connections 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- 172.21.15.49:0/2822716183 wait complete. 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- start start 2024-03-16T08:17:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 Processor -- start 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d89540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d89fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041a98b0 con 0x562ac7d787a0 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.053+0000 7f7b0e370740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041aad20 con 0x562ac7d85a60 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0e370740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041a9f80 con 0x562ac7d77e80 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0e370740 10 client.0 _ll_register_callbacks cb 0x7f7af40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50864/0 (socket says 172.21.15.49:50864) 2024-03-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 -- 172.21.15.49:0/1308227355 learned_addr learned my addr 172.21.15.49:0/1308227355 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b09c4b640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d89fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0944a640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d89540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 -- 172.21.15.49:0/1308227355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 msgr2=0x562ac7d89540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d89540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 -- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 msgr2=0x562ac7d89fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d89fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 -- 172.21.15.49:0/1308227355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac79b00f0 con 0x562ac7d787a0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b08c49640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7af4014010 tx=0x7f7af4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b38472b6ab0951e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7aecff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af402e070 con 0x562ac7d787a0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7af4005040 con 0x562ac7d787a0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af4025070 con 0x562ac7d787a0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7af403a050 con 0x562ac7d787a0 2024-03-16T08:17:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0e370740 10 client.15642 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.054+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ac7ca39d0 con 0x562ac7d787a0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af4025070 con 0x562ac7d787a0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7af4004700 con 0x562ac7d787a0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 10 client.15642 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 20 client.15642 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7dfbfb0 0x562ac7dfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 4 client.15642 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ac7ce3d90 con 0x562ac7dfbfb0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0944a640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7dfbfb0 0x562ac7dfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7aecff9640 10 client.15642 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.055+0000 7f7b0944a640 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7dfbfb0 0x562ac7dfe3a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b74f97c4b7b314e server_cookie=f6397d7097d2a2b8 in_seq=0 out_seq=0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7aecff9640 1 -- 172.21.15.49:0/1308227355 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ac7ce3d90 con 0x562ac7dfbfb0 2024-03-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7aecff9640 10 client.15642 handle_command_reply: tid=0 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 client.15642 shutdown 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 msgr2=0x562ac7d89a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7af4014010 tx=0x7f7af4005d10 comp rx=0 tx=0).stop 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 shutdown_connections 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562ac7dfbfb0 0x562ac7dfe3a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ac7d85a60 0x562ac7d89fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ac7d787a0 0x562ac7d89a80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 --2- 172.21.15.49:0/1308227355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac7d77e80 0x562ac7d89540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7aecff9640 0 client.15642 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 >> 172.21.15.49:0/1308227355 conn(0x562ac7d72880 msgr2=0x562ac7d73c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 shutdown_connections 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 1 -- 172.21.15.49:0/1308227355 wait complete. 2024-03-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0e370740 20 client.15642 trim_cache size 0 max 16384 2024-03-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 msgr2=0x7f7af00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 0x7f7af00a4ec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005cc0 tx=0x7f7afc032340 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 msgr2=0x7f7b04198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000d950 tx=0x7f7b0000de10 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 shutdown_connections 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7af00a2a00 0x7f7af00a4ec0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b04107760 0x7f7b0419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04102110 0x7f7b04198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 --2- 172.21.15.49:0/2940833973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b04101750 0x7f7b04198050 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.056+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 >> 172.21.15.49:0/2940833973 conn(0x7f7b040fd240 msgr2=0x7f7b040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.057+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 shutdown_connections 2024-03-16T08:17:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.057+0000 7f7b0b6ae640 1 -- 172.21.15.49:0/2940833973 wait complete. 2024-03-16T08:17:37.067 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:17:37.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- 172.21.15.49:0/3204115254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484101770 msgr2=0x7f8484101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 --2- 172.21.15.49:0/3204115254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484101770 0x7f8484101bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- 172.21.15.49:0/3204115254 shutdown_connections 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 --2- 172.21.15.49:0/3204115254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484102130 0x7f848410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 --2- 172.21.15.49:0/3204115254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484101770 0x7f8484101bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 --2- 172.21.15.49:0/3204115254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 0x7f8484107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- 172.21.15.49:0/3204115254 >> 172.21.15.49:0/3204115254 conn(0x7f84840fd280 msgr2=0x7f84840ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- 172.21.15.49:0/3204115254 shutdown_connections 2024-03-16T08:17:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- 172.21.15.49:0/3204115254 wait complete. 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 Processor -- start 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.176+0000 7f848b227640 1 -- start start 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 0x7f8484198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 0x7f848419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8484111b50 con 0x7f8484101770 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84841119d0 con 0x7f8484102130 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f848b227640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8484111cd0 con 0x7f8484107780 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50868/0 (socket says 172.21.15.49:50868) 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 -- 172.21.15.49:0/676583611 learned_addr learned my addr 172.21.15.49:0/676583611 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8483fff640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 0x7f8484198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f84897c4640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 0x7f848419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 -- 172.21.15.49:0/676583611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 msgr2=0x7f848419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 0x7f848419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 -- 172.21.15.49:0/676583611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 msgr2=0x7f8484198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 0x7f8484198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 -- 172.21.15.49:0/676583611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848419d040 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8483fff640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 0x7f8484198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8488fc3640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f847800ea60 tx=0x7f847800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.177+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847800cde0 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.178+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848419d330 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.178+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84780035f0 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.178+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478010670 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.178+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841a4bc0 con 0x7f8484101770 2024-03-16T08:17:37.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.179+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8484068a20 con 0x7f8484101770 2024-03-16T08:17:37.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.180+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f84780037a0 con 0x7f8484101770 2024-03-16T08:17:37.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.180+0000 7f8481ffb640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 0x7f84740a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.180+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f84780cf270 con 0x7f8484101770 2024-03-16T08:17:37.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.180+0000 7f8483fff640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 0x7f84740a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.181+0000 7f8483fff640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 0x7f84740a4ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f8470005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.185+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f847809beb0 con 0x7f8484101770 2024-03-16T08:17:37.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.410+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8484109d30 con 0x7f8484101770 2024-03-16T08:17:37.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.411+0000 7f8481ffb640 1 -- 172.21.15.49:0/676583611 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f847809b7b0 con 0x7f8484101770 2024-03-16T08:17:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:17:37.411 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:17:37.412 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:17:37.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 msgr2=0x7f84740a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 0x7f84740a4ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f8470005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 msgr2=0x7f8484198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f847800ea60 tx=0x7f847800ef20 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 shutdown_connections 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f84740a2a00 0x7f84740a4ec0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8484107780 0x7f848419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8484102130 0x7f8484198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 --2- 172.21.15.49:0/676583611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484101770 0x7f8484198050 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 >> 172.21.15.49:0/676583611 conn(0x7f84840fd280 msgr2=0x7f84840fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 shutdown_connections 2024-03-16T08:17:37.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.413+0000 7f848b227640 1 -- 172.21.15.49:0/676583611 wait complete. 2024-03-16T08:17:37.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:37 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:17:37.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:37 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:37 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:17:37.424 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:37 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.424 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:37 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:17:37.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:17:37.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:37 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:17:37.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:37 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:37 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:17:37.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:37 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:37 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:17:37.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 -- 172.21.15.49:0/2505240701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc103370 msgr2=0x7f52bc10b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/2505240701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc103370 0x7f52bc10b850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009900 tx=0x7f52a802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 -- 172.21.15.49:0/2505240701 shutdown_connections 2024-03-16T08:17:37.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/2505240701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc10be20 0x7f52bc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/2505240701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc103370 0x7f52bc10b850 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/2505240701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc102a50 0x7f52bc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 -- 172.21.15.49:0/2505240701 >> 172.21.15.49:0/2505240701 conn(0x7f52bc077840 msgr2=0x7f52bc077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 -- 172.21.15.49:0/2505240701 shutdown_connections 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.535+0000 7f52c2e3d640 1 -- 172.21.15.49:0/2505240701 wait complete. 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 Processor -- start 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 -- start start 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc102a50 0x7f52bc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc103370 0x7f52bc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc111bf0 con 0x7f52bc10be20 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc111a70 con 0x7f52bc103370 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c2e3d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc111d70 con 0x7f52bc102a50 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50884/0 (socket says 172.21.15.49:50884) 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 -- 172.21.15.49:0/1644926618 learned_addr learned my addr 172.21.15.49:0/1644926618 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc103370 0x7f52bc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc102a50 0x7f52bc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 -- 172.21.15.49:0/1644926618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc102a50 msgr2=0x7f52bc198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc102a50 0x7f52bc198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 -- 172.21.15.49:0/1644926618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc103370 msgr2=0x7f52bc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc103370 0x7f52bc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 -- 172.21.15.49:0/1644926618 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc19d0b0 con 0x7f52bc10be20 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.536+0000 7f52c13da640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52b400eb10 tx=0x7f52b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.537+0000 7f52b9ffb640 1 -- 172.21.15.49:0/1644926618 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004b20 con 0x7f52bc10be20 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.537+0000 7f52b9ffb640 1 -- 172.21.15.49:0/1644926618 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4004cc0 con 0x7f52bc10be20 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.537+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc19d3a0 con 0x7f52bc10be20 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.537+0000 7f52b9ffb640 1 -- 172.21.15.49:0/1644926618 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010760 con 0x7f52bc10be20 2024-03-16T08:17:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.537+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a4c30 con 0x7f52bc10be20 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.539+0000 7f52b9ffb640 1 -- 172.21.15.49:0/1644926618 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f52b4005020 con 0x7f52bc10be20 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.539+0000 7f52c5a21740 1 Processor -- start 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 -- start start 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52b9ffb640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 0x7f52ac0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8061490 con 0x55ebd80fd830 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8009c90 con 0x55ebd80fe150 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c5a21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebd805d530 con 0x55ebd8106d30 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 0x7f52ac0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52b9ffb640 1 -- 172.21.15.49:0/1644926618 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f52b40cf600 con 0x7f52bc10be20 2024-03-16T08:17:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50900/0 (socket says 172.21.15.49:50900) 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 -- 172.21.15.49:0/310627074 learned_addr learned my addr 172.21.15.49:0/310627074 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52bbfff640 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c13da640 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 -- 172.21.15.49:0/310627074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 msgr2=0x55ebd8109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 -- 172.21.15.49:0/310627074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 msgr2=0x55ebd81067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 -- 172.21.15.49:0/310627074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd80c4500 con 0x55ebd80fd830 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009cf0 tx=0x7f52b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30be210a61dea1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.540+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 0x7f52ac0a4ec0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52a802efd0 tx=0x7f52a802ca20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52a6ffd640 1 -- 172.21.15.49:0/310627074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004230 con 0x55ebd80fd830 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52a6ffd640 1 -- 172.21.15.49:0/310627074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b00043d0 con 0x55ebd80fd830 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52a6ffd640 1 -- 172.21.15.49:0/310627074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0019840 con 0x55ebd80fd830 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 -- 172.21.15.49:0/310627074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 msgr2=0x55ebd80fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009cf0 tx=0x7f52b0011810 comp rx=0 tx=0).stop 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 -- 172.21.15.49:0/310627074 shutdown_connections 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 --2- 172.21.15.49:0/310627074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd80fdc10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 -- 172.21.15.49:0/310627074 >> 172.21.15.49:0/310627074 conn(0x55ebd80f7f60 msgr2=0x55ebd80f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 -- 172.21.15.49:0/310627074 shutdown_connections 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.541+0000 7f52c5a21740 1 -- 172.21.15.49:0/310627074 wait complete. 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 -- start start 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 Processor -- start 2024-03-16T08:17:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd8103d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd81047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8109b20 con 0x55ebd80fe150 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ebd801d230 con 0x55ebd80fd830 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8109ca0 con 0x55ebd8106d30 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c5a21740 10 client.0 _ll_register_callbacks cb 0x55ebd8059010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd8103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50908/0 (socket says 172.21.15.49:50908) 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52bbfff640 1 -- 172.21.15.49:0/1763212462 learned_addr learned my addr 172.21.15.49:0/1763212462 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.542+0000 7f52c13da640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd81047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 -- 172.21.15.49:0/1763212462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 msgr2=0x55ebd81047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd81047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 -- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 msgr2=0x55ebd8103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd8103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 -- 172.21.15.49:0/1763212462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd802c100 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52bbfff640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009900 tx=0x7f52a802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1823b5153ffcb321 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005110 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f52a80052b0 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b00043d0 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52a805b050 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52c5a21740 10 client.15660 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ebd7d753f0 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b00043d0 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.543+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52a804d450 con 0x55ebd80fe150 2024-03-16T08:17:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 10 client.15660 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 20 client.15660 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd81170d0 0x55ebd8119590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 4 client.15660 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd806e9d0 con 0x55ebd81170d0 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd81170d0 0x55ebd8119590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f529bfff640 10 client.15660 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.544+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd81170d0 0x55ebd8119590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f9ecf6b480e5670 server_cookie=e34941482f987216 in_seq=0 out_seq=0 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f529bfff640 1 -- 172.21.15.49:0/1763212462 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ebd806e9d0 con 0x55ebd81170d0 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f529bfff640 10 client.15660 handle_command_reply: tid=0 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 client.15660 shutdown 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 msgr2=0x55ebd81042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009900 tx=0x7f52a802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 shutdown_connections 2024-03-16T08:17:37.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd81170d0 0x55ebd8119590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f529bfff640 0 client.15660 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd81042b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 --2- 172.21.15.49:0/1763212462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.545+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 >> 172.21.15.49:0/1763212462 conn(0x55ebd80f7f60 msgr2=0x55ebd80f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.546+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 shutdown_connections 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.546+0000 7f52c5a21740 1 -- 172.21.15.49:0/1763212462 wait complete. 2024-03-16T08:17:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.546+0000 7f52c5a21740 20 client.15660 trim_cache size 0 max 16384 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 Processor -- start 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 -- start start 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd811ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1afd20 con 0x55ebd80fd830 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ad5f0 con 0x55ebd80fe150 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c5a21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ab420 con 0x55ebd8106d30 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50910/0 (socket says 172.21.15.49:50910) 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c0bd9640 1 -- 172.21.15.49:0/2011327045 learned_addr learned my addr 172.21.15.49:0/2011327045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c13da640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd811ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52bbfff640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c0bd9640 1 -- 172.21.15.49:0/2011327045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 msgr2=0x55ebd811ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.559+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd811ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c0bd9640 1 -- 172.21.15.49:0/2011327045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 msgr2=0x55ebd81128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c0bd9640 1 -- 172.21.15.49:0/2011327045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd80feb00 con 0x55ebd80fd830 2024-03-16T08:17:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52bbfff640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005fd0 tx=0x7f52b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c302fea2ea7fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52a67fc640 1 -- 172.21.15.49:0/2011327045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002ba30 con 0x55ebd80fd830 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52a67fc640 1 -- 172.21.15.49:0/2011327045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b0030070 con 0x55ebd80fd830 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52a67fc640 1 -- 172.21.15.49:0/2011327045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0005330 con 0x55ebd80fd830 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 -- 172.21.15.49:0/2011327045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 msgr2=0x55ebd81064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005fd0 tx=0x7f52b0011810 comp rx=0 tx=0).stop 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 -- 172.21.15.49:0/2011327045 shutdown_connections 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd811ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fe150 0x55ebd81128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 --2- 172.21.15.49:0/2011327045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fd830 0x55ebd81064a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.560+0000 7f52c5a21740 1 -- 172.21.15.49:0/2011327045 >> 172.21.15.49:0/2011327045 conn(0x55ebd80f7f60 msgr2=0x55ebd80f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- 172.21.15.49:0/2011327045 shutdown_connections 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- 172.21.15.49:0/2011327045 wait complete. 2024-03-16T08:17:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- start start 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 Processor -- start 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd810f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1abab0 con 0x55ebd80fe150 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1acf20 con 0x55ebd80fd830 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ac180 con 0x55ebd8106d30 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c5a21740 10 client.0 _ll_register_callbacks cb 0x7f52b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50918/0 (socket says 172.21.15.49:50918) 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52bbfff640 1 -- 172.21.15.49:0/3085058174 learned_addr learned my addr 172.21.15.49:0/3085058174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c0bd9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd810f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.561+0000 7f52c13da640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 -- 172.21.15.49:0/3085058174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 msgr2=0x55ebd8100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 -- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 msgr2=0x55ebd810f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd810f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 -- 172.21.15.49:0/3085058174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd7e3f7d0 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52c13da640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52bbfff640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f52a804d8f0 tx=0x7f52a804dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe74fb8b6aa2fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f529bfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005110 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f52a80052b0 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b0030070 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52a804d660 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52c5a21740 10 client.15666 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ebd817d360 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.562+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b0030070 con 0x55ebd80fe150 2024-03-16T08:17:37.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52a8003ad0 con 0x55ebd80fe150 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 10 client.15666 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 20 client.15666 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd817da00 0x55ebd817fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 4 client.15666 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebd8068d20 con 0x55ebd817da00 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd817da00 0x55ebd817fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f529bfff640 10 client.15666 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c0bd9640 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd817da00 0x55ebd817fe20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ab62c57515baff9 server_cookie=8b1c034f8bccfdb7 in_seq=0 out_seq=0 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f529bfff640 1 -- 172.21.15.49:0/3085058174 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebd8068d20 con 0x55ebd817da00 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f529bfff640 10 client.15666 handle_command_reply: tid=0 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 1 client.15666 shutdown 2024-03-16T08:17:37.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 msgr2=0x55ebd8100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.563+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f52a804d8f0 tx=0x7f52a804dd10 comp rx=0 tx=0).stop 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 shutdown_connections 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55ebd817da00 0x55ebd817fe20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebd8106d30 0x55ebd8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ebd80fe150 0x55ebd8100680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 --2- 172.21.15.49:0/3085058174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ebd80fd830 0x55ebd810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f529bfff640 0 client.15666 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 >> 172.21.15.49:0/3085058174 conn(0x55ebd80f7f60 msgr2=0x55ebd80f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 shutdown_connections 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 1 -- 172.21.15.49:0/3085058174 wait complete. 2024-03-16T08:17:37.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c5a21740 20 client.15666 trim_cache size 0 max 16384 2024-03-16T08:17:37.572 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:37.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 msgr2=0x7f52ac0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 0x7f52ac0a4ec0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52a802efd0 tx=0x7f52a802ca20 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 msgr2=0x7f52bc19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52b400eb10 tx=0x7f52b400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 shutdown_connections 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f52ac0a2a00 0x7f52ac0a4ec0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52bc10be20 0x7f52bc19c980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52bc103370 0x7f52bc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 --2- 172.21.15.49:0/1644926618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52bc102a50 0x7f52bc198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 >> 172.21.15.49:0/1644926618 conn(0x7f52bc077840 msgr2=0x7f52bc0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 shutdown_connections 2024-03-16T08:17:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.564+0000 7f52c2e3d640 1 -- 172.21.15.49:0/1644926618 wait complete. 2024-03-16T08:17:37.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:17:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:37 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:17:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:37 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:37 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:17:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:37 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:37.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:37 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:17:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.685+0000 7fe0c903f640 1 -- 172.21.15.49:0/450233556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4101780 msgr2=0x7fe0c4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 --2- 172.21.15.49:0/450233556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4101780 0x7fe0c4101c00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009930 tx=0x7fe0b802bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 -- 172.21.15.49:0/450233556 shutdown_connections 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 --2- 172.21.15.49:0/450233556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4102140 0x7fe0c410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 --2- 172.21.15.49:0/450233556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4101780 0x7fe0c4101c00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 --2- 172.21.15.49:0/450233556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 0x7fe0c4107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 -- 172.21.15.49:0/450233556 >> 172.21.15.49:0/450233556 conn(0x7fe0c40fd210 msgr2=0x7fe0c40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 -- 172.21.15.49:0/450233556 shutdown_connections 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 -- 172.21.15.49:0/450233556 wait complete. 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.686+0000 7fe0c903f640 1 Processor -- start 2024-03-16T08:17:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 -- start start 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 0x7fe0c419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 0x7fe0c419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4111b60 con 0x7fe0c4102140 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41119e0 con 0x7fe0c4101780 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c903f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4111ce0 con 0x7fe0c4107790 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50944/0 (socket says 172.21.15.49:50944) 2024-03-16T08:17:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 -- 172.21.15.49:0/3507237858 learned_addr learned my addr 172.21.15.49:0/3507237858 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 0x7fe0c419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c359e640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 0x7fe0c419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 -- 172.21.15.49:0/3507237858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 msgr2=0x7fe0c419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 0x7fe0c419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 -- 172.21.15.49:0/3507237858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 msgr2=0x7fe0c419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 0x7fe0c419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 -- 172.21.15.49:0/3507237858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c419f2b0 con 0x7fe0c4102140 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 0x7fe0c419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.687+0000 7fe0c259c640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009230 tx=0x7fe0b802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.688+0000 7fe0abfff640 1 -- 172.21.15.49:0/3507237858 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8041070 con 0x7fe0c4102140 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.688+0000 7fe0abfff640 1 -- 172.21.15.49:0/3507237858 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b8004450 con 0x7fe0c4102140 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.688+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c419f540 con 0x7fe0c4102140 2024-03-16T08:17:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.688+0000 7fe0abfff640 1 -- 172.21.15.49:0/3507237858 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8003de0 con 0x7fe0c4102140 2024-03-16T08:17:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.688+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41a6e30 con 0x7fe0c4102140 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0abfff640 1 -- 172.21.15.49:0/3507237858 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fe0b8044070 con 0x7fe0c4102140 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0cbc20740 1 Processor -- start 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0abfff640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 0x7fe0ac0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0cbc20740 1 -- start start 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0abfff640 1 -- 172.21.15.49:0/3507237858 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe0b80f07e0 con 0x7fe0c4102140 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.690+0000 7fe0c359e640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 0x7fe0ac0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f66d30 0x55f607f69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f607ec1490 con 0x55f607f5e150 2024-03-16T08:17:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f607e69c90 con 0x55f607f5d830 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f607ebd530 con 0x55f607f66d30 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50950/0 (socket says 172.21.15.49:50950) 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:38348/0 (socket says 172.21.15.49:38348) 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 -- 172.21.15.49:0/2351074134 learned_addr learned my addr 172.21.15.49:0/2351074134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c359e640 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f66d30 0x55f607f69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 -- 172.21.15.49:0/2351074134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f66d30 msgr2=0x55f607f69120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f66d30 0x55f607f69120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c359e640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 0x7fe0ac0a4ec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005fd0 tx=0x7fe0bc011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 -- 172.21.15.49:0/2351074134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 msgr2=0x55f607f5dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 -- 172.21.15.49:0/2351074134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f607f24500 con 0x55f607f5e150 2024-03-16T08:17:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c14ea82d20d3153 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0a97fa640 1 -- 172.21.15.49:0/2351074134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b804b040 con 0x55f607f5e150 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0a97fa640 1 -- 172.21.15.49:0/2351074134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b80bd530 con 0x55f607f5e150 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.691+0000 7fe0a97fa640 1 -- 172.21.15.49:0/2351074134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80b94a0 con 0x55f607f5e150 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2351074134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 msgr2=0x55f607f667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b8009260 comp rx=0 tx=0).stop 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2351074134 shutdown_connections 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f66d30 0x55f607f69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5e150 0x55f607f667f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2351074134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f5dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2351074134 >> 172.21.15.49:0/2351074134 conn(0x55f607f57f60 msgr2=0x55f607f58340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2351074134 shutdown_connections 2024-03-16T08:17:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2351074134 wait complete. 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.692+0000 7fe0cbc20740 1 -- start start 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 Processor -- start 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f64270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f607f698e0 con 0x55f607f5d830 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f607e7d230 con 0x55f607f66d30 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f607f69a60 con 0x55f607f5e150 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0cbc20740 10 client.0 _ll_register_callbacks cb 0x55f607eb9010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50960/0 (socket says 172.21.15.49:50960) 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/93560778 learned_addr learned my addr 172.21.15.49:0/93560778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c259c640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f64270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c359e640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 msgr2=0x55f607f64270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f64270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/93560778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 msgr2=0x55f607f647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/93560778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f607e8c100 con 0x55f607f5d830 2024-03-16T08:17:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4012d20 tx=0x7fe0b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987214b07192b46f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.693+0000 7fe0a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40120d0 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe0b4010430 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b80bd530 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe0b4012270 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 10 client.15678 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f607bd53f0 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b80bd530 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe0b401c750 con 0x55f607f5d830 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 10 client.15678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 20 client.15678 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607f76e50 0x55f607f79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 4 client.15678 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f607ece9d0 con 0x55f607f76e50 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.694+0000 7fe0c259c640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607f76e50 0x55f607f79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.695+0000 7fe0c259c640 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607f76e50 0x55f607f79260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=674646005023ee2d server_cookie=4efe1d0d5e53a607 in_seq=0 out_seq=0 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.695+0000 7fe0a6ff5640 10 client.15678 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.695+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/93560778 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f607ece9d0 con 0x55f607f76e50 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.695+0000 7fe0a6ff5640 10 client.15678 handle_command_reply: tid=0 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 client.15678 shutdown 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 msgr2=0x55f607f63d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4012d20 tx=0x7fe0b400f790 comp rx=0 tx=0).stop 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 shutdown_connections 2024-03-16T08:17:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607f76e50 0x55f607f79260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f64270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0a6ff5640 0 client.15678 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/93560778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f63d30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 >> 172.21.15.49:0/93560778 conn(0x55f607f57f60 msgr2=0x55f607f59cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 shutdown_connections 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 1 -- 172.21.15.49:0/93560778 wait complete. 2024-03-16T08:17:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.696+0000 7fe0cbc20740 20 client.15678 trim_cache size 0 max 16384 2024-03-16T08:17:37.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 Processor -- start 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 -- start start 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f7eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b1f10 con 0x55f607f5d830 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41af7e0 con 0x55f607f66d30 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ad610 con 0x55f607f5e150 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50974/0 (socket says 172.21.15.49:50974) 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/2731333014 learned_addr learned my addr 172.21.15.49:0/2731333014 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0c359e640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f7eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.709+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/2731333014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 msgr2=0x55f607f645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/2731333014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 msgr2=0x55f607f7eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f7eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/2731333014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f607c9f7d0 con 0x55f607f5d830 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe0b401cbc0 tx=0x7fe0b400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686c4c7e1a7449c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0a8ff9640 1 -- 172.21.15.49:0/2731333014 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4029070 con 0x55f607f5d830 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0a8ff9640 1 -- 172.21.15.49:0/2731333014 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b401c900 con 0x55f607f5d830 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0a8ff9640 1 -- 172.21.15.49:0/2731333014 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40048a0 con 0x55f607f5d830 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2731333014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 msgr2=0x55f607f72500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe0b401cbc0 tx=0x7fe0b400fd10 comp rx=0 tx=0).stop 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2731333014 shutdown_connections 2024-03-16T08:17:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f66d30 0x55f607f7eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.710+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2731333014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f5d830 0x55f607f72500 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2731333014 >> 172.21.15.49:0/2731333014 conn(0x55f607f57f60 msgr2=0x55f607f56fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2731333014 shutdown_connections 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2731333014 wait complete. 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- start start 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 Processor -- start 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f6eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f69ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f66d30 0x55f607f6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41adca0 con 0x55f607f66d30 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41af110 con 0x55f607f5d830 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ae370 con 0x55f607f5e150 2024-03-16T08:17:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0cbc20740 10 client.0 _ll_register_callbacks cb 0x7fe0b4009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f6eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0c259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f69ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0c359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f66d30 0x55f607f6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f6eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:38368/0 (socket says 172.21.15.49:38368) 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.711+0000 7fe0c2d9d640 1 -- 172.21.15.49:0/2134086245 learned_addr learned my addr 172.21.15.49:0/2134086245 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 -- 172.21.15.49:0/2134086245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 msgr2=0x55f607f6eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f6eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 -- 172.21.15.49:0/2134086245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f66d30 msgr2=0x55f607f6a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f66d30 0x55f607f6a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 -- 172.21.15.49:0/2134086245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f607f6a920 con 0x55f607f5e150 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0c259c640 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f69ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80c1140 tx=0x7fe0b8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719d2f77c6bbdd79 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:17:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b809f070 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b401c900 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe0b80b94a0 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe0b804b040 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 10 client.25564 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f607fdce40 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b401c900 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe0b80c5140 con 0x55f607f5e150 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 10 client.25564 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 10 client.25564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.712+0000 7fe0cbc20740 20 client.25564 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607fdd3d0 0x55f607fdf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 4 client.25564 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f607ec8d20 con 0x55f607fdd3d0 2024-03-16T08:17:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607fdd3d0 0x55f607fdf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0a6ff5640 10 client.25564 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0c2d9d640 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607fdd3d0 0x55f607fdf7c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fda179a7f1f1f8dc server_cookie=5bde04532a655f78 in_seq=0 out_seq=0 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0a6ff5640 1 -- 172.21.15.49:0/2134086245 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f607ec8d20 con 0x55f607fdd3d0 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0a6ff5640 10 client.25564 handle_command_reply: tid=0 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 client.25564 shutdown 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 msgr2=0x55f607f69ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f69ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80c1140 tx=0x7fe0b8005fd0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 shutdown_connections 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55f607fdd3d0 0x55f607fdf7c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f607f66d30 0x55f607f6a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f607f5e150 0x55f607f69ea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 --2- 172.21.15.49:0/2134086245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f607f5d830 0x55f607f6eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0a6ff5640 0 client.25564 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:17:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 >> 172.21.15.49:0/2134086245 conn(0x55f607f57f60 msgr2=0x55f607f59120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 shutdown_connections 2024-03-16T08:17:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.713+0000 7fe0cbc20740 1 -- 172.21.15.49:0/2134086245 wait complete. 2024-03-16T08:17:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0cbc20740 20 client.25564 trim_cache size 0 max 16384 2024-03-16T08:17:37.721 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 msgr2=0x7fe0ac0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 0x7fe0ac0a4ec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005fd0 tx=0x7fe0bc011810 comp rx=0 tx=0).stop 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 msgr2=0x7fe0c419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009230 tx=0x7fe0b802cb30 comp rx=0 tx=0).stop 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 shutdown_connections 2024-03-16T08:17:37.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe0ac0a2a00 0x7fe0ac0a4ec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0c4107790 0x7fe0c419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c4102140 0x7fe0c419a800 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 --2- 172.21.15.49:0/3507237858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4101780 0x7fe0c419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 >> 172.21.15.49:0/3507237858 conn(0x7fe0c40fd210 msgr2=0x7fe0c410ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 shutdown_connections 2024-03-16T08:17:37.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.714+0000 7fe0c903f640 1 -- 172.21.15.49:0/3507237858 wait complete. 2024-03-16T08:17:37.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:17:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.835+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/3253195798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac107790 msgr2=0x7fe4ac107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.835+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/3253195798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac107790 0x7fe4ac107b70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009930 tx=0x7fe49c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/3253195798 shutdown_connections 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/3253195798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac102140 0x7fe4ac10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/3253195798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac101780 0x7fe4ac101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/3253195798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac107790 0x7fe4ac107b70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/3253195798 >> 172.21.15.49:0/3253195798 conn(0x7fe4ac0fd210 msgr2=0x7fe4ac0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/3253195798 shutdown_connections 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/3253195798 wait complete. 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 Processor -- start 2024-03-16T08:17:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.836+0000 7fe4b1c4b640 1 -- start start 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac101780 0x7fe4ac073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac107790 0x7fe4ac074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111ba0 con 0x7fe4ac102140 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111a20 con 0x7fe4ac101780 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111d20 con 0x7fe4ac107790 2024-03-16T08:17:37.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51000/0 (socket says 172.21.15.49:51000) 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/1192422532 learned_addr learned my addr 172.21.15.49:0/1192422532 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac101780 0x7fe4ac073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4abfff640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac107790 0x7fe4ac074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/1192422532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac107790 msgr2=0x7fe4ac074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac107790 0x7fe4ac074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/1192422532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac101780 msgr2=0x7fe4ac073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac101780 0x7fe4ac073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/1192422532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac1a65c0 con 0x7fe4ac102140 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe49400e9f0 tx=0x7fe49400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a97fa640 1 -- 172.21.15.49:0/1192422532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49400ccc0 con 0x7fe4ac102140 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac1a68b0 con 0x7fe4ac102140 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a97fa640 1 -- 172.21.15.49:0/1192422532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49400ce60 con 0x7fe4ac102140 2024-03-16T08:17:37.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4a97fa640 1 -- 172.21.15.49:0/1192422532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494010690 con 0x7fe4ac102140 2024-03-16T08:17:37.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.837+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1a6e00 con 0x7fe4ac102140 2024-03-16T08:17:37.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.839+0000 7fe4a97fa640 1 -- 172.21.15.49:0/1192422532 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fe4940046c0 con 0x7fe4ac102140 2024-03-16T08:17:37.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 Processor -- start 2024-03-16T08:17:37.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 -- start start 2024-03-16T08:17:37.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c77cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c788120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55916c6e0490 con 0x55916c77c830 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55916c688c90 con 0x55916c785d30 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4b4836740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55916c6dc530 con 0x55916c77d150 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4a97fa640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 0x7fe4980a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4a2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c788120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c77cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:52012/0 (socket says 172.21.15.49:52012) 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/749883791 learned_addr learned my addr 172.21.15.49:0/749883791 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.840+0000 7fe4abfff640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 0x7fe4980a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/749883791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 msgr2=0x55916c788120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c788120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4a97fa640 1 -- 172.21.15.49:0/1192422532 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe4940cf370 con 0x7fe4ac102140 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/749883791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 msgr2=0x55916c77cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c77cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/749883791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916c743500 con 0x55916c77d150 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4abfff640 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c77cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4abfff640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 0x7fe4980a4ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005fd0 tx=0x7fe4a4009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02cfd0 tx=0x7fe49c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edfdeb75bae9874 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4a1ffb640 1 -- 172.21.15.49:0/749883791 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c042070 con 0x55916c77d150 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4a1ffb640 1 -- 172.21.15.49:0/749883791 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c004030 con 0x55916c77d150 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.841+0000 7fe4a1ffb640 1 -- 172.21.15.49:0/749883791 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004bc0 con 0x55916c77d150 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- 172.21.15.49:0/749883791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 msgr2=0x55916c7857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02cfd0 tx=0x7fe49c0095f0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- 172.21.15.49:0/749883791 shutdown_connections 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c788120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c7857f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 --2- 172.21.15.49:0/749883791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c77cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- 172.21.15.49:0/749883791 >> 172.21.15.49:0/749883791 conn(0x55916c776f60 msgr2=0x55916c777340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- 172.21.15.49:0/749883791 shutdown_connections 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- 172.21.15.49:0/749883791 wait complete. 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 -- start start 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.842+0000 7fe4b4836740 1 Processor -- start 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 0x55916c7832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55916c788ac0 con 0x55916c785d30 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55916c69c230 con 0x55916c77d150 2024-03-16T08:17:37.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55916c788c40 con 0x55916c77c830 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4b4836740 10 client.0 _ll_register_callbacks cb 0x55916c6d8010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:52024/0 (socket says 172.21.15.49:52024) 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51020/0 (socket says 172.21.15.49:51020) 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2091125146 learned_addr learned my addr 172.21.15.49:0/2091125146 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 0x55916c7832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 msgr2=0x55916c7832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 0x55916c7832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2091125146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 msgr2=0x55916c783830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2091125146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916c6ab100 con 0x55916c77c830 2024-03-16T08:17:37.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4abfff640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 0x55916c7832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02c170 tx=0x7fe49c004bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e20745e50f06c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.843+0000 7fe48affd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c042040 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49c004030 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe49c02ca90 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe49c02cc90 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 10 client.25576 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55916c3f43f0 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c004030 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe49c003850 con 0x55916c77c830 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 10 client.25576 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 20 client.25576 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c795f30 0x55916c798320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 4 client.25576 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916c6ed9d0 con 0x55916c795f30 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.844+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c795f30 0x55916c798320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.845+0000 7fe48affd640 10 client.25576 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.845+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c795f30 0x55916c798320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f909678dccb96aa2 server_cookie=a6fcb272945ee513 in_seq=0 out_seq=0 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.845+0000 7fe48affd640 1 -- 172.21.15.49:0/2091125146 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55916c6ed9d0 con 0x55916c795f30 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.845+0000 7fe48affd640 10 client.25576 handle_command_reply: tid=0 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.845+0000 7fe4b4836740 1 client.25576 shutdown 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 msgr2=0x55916c782db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02c170 tx=0x7fe49c004bf0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 shutdown_connections 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c795f30 0x55916c798320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c785d30 0x55916c783830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c77d150 0x55916c7832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2091125146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c782db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe48affd640 0 client.25576 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 >> 172.21.15.49:0/2091125146 conn(0x55916c776f60 msgr2=0x55916c778cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 shutdown_connections 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 1 -- 172.21.15.49:0/2091125146 wait complete. 2024-03-16T08:17:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.846+0000 7fe4b4836740 20 client.25576 trim_cache size 0 max 16384 2024-03-16T08:17:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 Processor -- start 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 -- start start 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c778a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c79b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b1dc0 con 0x55916c77c830 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1af690 con 0x55916c785d30 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4b4836740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1ad4c0 con 0x55916c77d150 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51034/0 (socket says 172.21.15.49:51034) 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2981068870 learned_addr learned my addr 172.21.15.49:0/2981068870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c778a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4abfff640 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c79b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.859+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2981068870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 msgr2=0x55916c778a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c778a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2981068870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 msgr2=0x55916c79b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c79b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4ab7fe640 1 -- 172.21.15.49:0/2981068870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916c77cf80 con 0x55916c77c830 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe49c005d50 tx=0x7fe49c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c229548d4639ef68 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4a17fa640 1 -- 172.21.15.49:0/2981068870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c042040 con 0x55916c77c830 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4a17fa640 1 -- 172.21.15.49:0/2981068870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c02c5d0 con 0x55916c77c830 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4a17fa640 1 -- 172.21.15.49:0/2981068870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004f50 con 0x55916c77c830 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 -- 172.21.15.49:0/2981068870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 msgr2=0x55916c7854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe49c005d50 tx=0x7fe49c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 -- 172.21.15.49:0/2981068870 shutdown_connections 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c79b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77d150 0x55916c778a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 --2- 172.21.15.49:0/2981068870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77c830 0x55916c7854e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 -- 172.21.15.49:0/2981068870 >> 172.21.15.49:0/2981068870 conn(0x55916c776f60 msgr2=0x55916c778340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 -- 172.21.15.49:0/2981068870 shutdown_connections 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.860+0000 7fe4b4836740 1 -- 172.21.15.49:0/2981068870 wait complete. 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 -- start start 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 Processor -- start 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c7f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c798500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1adb50 con 0x55916c77d150 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1aefc0 con 0x55916c785d30 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1ae220 con 0x55916c77c830 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4b4836740 10 client.0 _ll_register_callbacks cb 0x7fe4a4009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51050/0 (socket says 172.21.15.49:51050) 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c798500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/909371527 learned_addr learned my addr 172.21.15.49:0/909371527 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.861+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c7f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/909371527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 msgr2=0x55916c7f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c7f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 msgr2=0x55916c798500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c798500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 -- 172.21.15.49:0/909371527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916c4be7d0 con 0x55916c77d150 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4a2ffd640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d010 tx=0x7fe494097bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3a3b65d567cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49409b8b0 con 0x55916c77d150 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49c02c5d0 con 0x55916c77d150 2024-03-16T08:17:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe494099bb0 con 0x55916c77d150 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4940ba170 con 0x55916c77d150 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 10 client.15702 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55916c7fb8e0 con 0x55916c77d150 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c02c5d0 con 0x55916c77d150 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe4940a3bc0 con 0x55916c77d150 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 10 client.15702 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.862+0000 7fe4b4836740 20 client.15702 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c7fc010 0x55916c7fe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 4 client.15702 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55916c6e7d20 con 0x55916c7fc010 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c7fc010 0x55916c7fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4ab7fe640 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c7fc010 0x55916c7fe430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd65db692116d server_cookie=94fd890d4e6deac6 in_seq=0 out_seq=0 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe48affd640 10 client.15702 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe48affd640 1 -- 172.21.15.49:0/909371527 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55916c6e7d20 con 0x55916c7fc010 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe48affd640 10 client.15702 handle_command_reply: tid=0 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 client.15702 shutdown 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 msgr2=0x55916c7f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d010 tx=0x7fe494097bc0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 shutdown_connections 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55916c7fc010 0x55916c7fe430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55916c785d30 0x55916c798500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe48affd640 0 client.15702 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55916c77d150 0x55916c7f4800 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 --2- 172.21.15.49:0/909371527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55916c77c830 0x55916c7f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 >> 172.21.15.49:0/909371527 conn(0x55916c776f60 msgr2=0x55916c791720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 shutdown_connections 2024-03-16T08:17:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.863+0000 7fe4b4836740 1 -- 172.21.15.49:0/909371527 wait complete. 2024-03-16T08:17:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b4836740 20 client.15702 trim_cache size 0 max 16384 2024-03-16T08:17:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 msgr2=0x7fe4980a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 0x7fe4980a4ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005fd0 tx=0x7fe4a4009500 comp rx=0 tx=0).stop 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 msgr2=0x7fe4ac074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe49400e9f0 tx=0x7fe49400edf0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 shutdown_connections 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4980a2a00 0x7fe4980a4ec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac107790 0x7fe4ac074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4ac102140 0x7fe4ac074020 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 --2- 172.21.15.49:0/1192422532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4ac101780 0x7fe4ac073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 >> 172.21.15.49:0/1192422532 conn(0x7fe4ac0fd210 msgr2=0x7fe4ac0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 shutdown_connections 2024-03-16T08:17:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:37.864+0000 7fe4b1c4b640 1 -- 172.21.15.49:0/1192422532 wait complete. 2024-03-16T08:17:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:38 smithi154 ceph-mon[36471]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:38 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/676583611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:38.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:38 smithi049 ceph-mon[31942]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:38.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:38 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/676583611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:38.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:38 smithi129 ceph-mon[36561]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-16T08:17:38.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:38 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/676583611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:17:38.875 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:17:38.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:17:38.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:17:39.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 -- 172.21.15.49:0/2724919992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880100650 msgr2=0x7fe880100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 --2- 172.21.15.49:0/2724919992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880100650 0x7fe880100ad0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe874009930 tx=0x7fe87402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:39.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 -- 172.21.15.49:0/2724919992 shutdown_connections 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 --2- 172.21.15.49:0/2724919992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880101010 0x7fe88010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 --2- 172.21.15.49:0/2724919992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880100650 0x7fe880100ad0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 --2- 172.21.15.49:0/2724919992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 0x7fe880106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 -- 172.21.15.49:0/2724919992 >> 172.21.15.49:0/2724919992 conn(0x7fe8800777e0 msgr2=0x7fe880077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 -- 172.21.15.49:0/2724919992 shutdown_connections 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.001+0000 7fe88764b640 1 -- 172.21.15.49:0/2724919992 wait complete. 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 Processor -- start 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 -- start start 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880100650 0x7fe8801980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 0x7fe88019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe880110a70 con 0x7fe880101010 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801108f0 con 0x7fe880106660 2024-03-16T08:17:39.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe88764b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe880110bf0 con 0x7fe880100650 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51066/0 (socket says 172.21.15.49:51066) 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 -- 172.21.15.49:0/524053449 learned_addr learned my addr 172.21.15.49:0/524053449 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe8853e7640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880100650 0x7fe8801980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe885be8640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 0x7fe88019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 -- 172.21.15.49:0/524053449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880100650 msgr2=0x7fe8801980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880100650 0x7fe8801980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 -- 172.21.15.49:0/524053449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 msgr2=0x7fe88019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 0x7fe88019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.002+0000 7fe884be6640 1 -- 172.21.15.49:0/524053449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88019d090 con 0x7fe880101010 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe884be6640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe874009230 tx=0x7fe87402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe87e7fc640 1 -- 172.21.15.49:0/524053449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874041070 con 0x7fe880101010 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88019d320 con 0x7fe880101010 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe87e7fc640 1 -- 172.21.15.49:0/524053449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe874004450 con 0x7fe880101010 2024-03-16T08:17:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe87e7fc640 1 -- 172.21.15.49:0/524053449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874003de0 con 0x7fe880101010 2024-03-16T08:17:39.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.003+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8801a4c10 con 0x7fe880101010 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.005+0000 7fe87e7fc640 1 -- 172.21.15.49:0/524053449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7fe874044070 con 0x7fe880101010 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 Processor -- start 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 -- start start 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca401190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca3590d0 con 0x55c6ca3f6710 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca301c70 con 0x55c6ca3feda0 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe88a23b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca355170 con 0x55c6ca3f5a30 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe87e7fc640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 0x7fe8680a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe884be6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca401190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe885be8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:52080/0 (socket says 172.21.15.49:52080) 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 -- 172.21.15.49:0/1365659517 learned_addr learned my addr 172.21.15.49:0/1365659517 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe885be8640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 0x7fe8680a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 -- 172.21.15.49:0/1365659517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 msgr2=0x55c6ca401190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe87e7fc640 1 -- 172.21.15.49:0/524053449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe8740f07e0 con 0x7fe880101010 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca401190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 -- 172.21.15.49:0/1365659517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 msgr2=0x55c6ca3f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe8853e7640 1 -- 172.21.15.49:0/1365659517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ca1357e0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.006+0000 7fe885be8640 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.007+0000 7fe8853e7640 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe878009c70 tx=0x7fe878011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3821a65f74ae8d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.007+0000 7fe885be8640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 0x7fe8680a4ec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe870009b80 tx=0x7fe870009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.007+0000 7fe8677fe640 1 -- 172.21.15.49:0/1365659517 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878004230 con 0x55c6ca3f5a30 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.007+0000 7fe8677fe640 1 -- 172.21.15.49:0/1365659517 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8780043d0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.007+0000 7fe8677fe640 1 -- 172.21.15.49:0/1365659517 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780197b0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- 172.21.15.49:0/1365659517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 msgr2=0x55c6ca3f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe878009c70 tx=0x7fe878011810 comp rx=0 tx=0).stop 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- 172.21.15.49:0/1365659517 shutdown_connections 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca401190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca3f5eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- 172.21.15.49:0/1365659517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- 172.21.15.49:0/1365659517 >> 172.21.15.49:0/1365659517 conn(0x55c6ca3bcc50 msgr2=0x55c6ca3f0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- 172.21.15.49:0/1365659517 shutdown_connections 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- 172.21.15.49:0/1365659517 wait complete. 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 -- start start 2024-03-16T08:17:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 Processor -- start 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f6710 0x55c6ca4095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.008+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe88a23b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca401c10 con 0x55c6ca3f5a30 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe88a23b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca3150d0 con 0x55c6ca3feda0 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe88a23b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c6ca401d90 con 0x55c6ca3f6710 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe88a23b740 10 client.0 _ll_register_callbacks cb 0x55c6ca2de760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51086/0 (socket says 172.21.15.49:51086) 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 -- 172.21.15.49:0/181854269 learned_addr learned my addr 172.21.15.49:0/181854269 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe884be6640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f6710 0x55c6ca4095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe885be8640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 -- 172.21.15.49:0/181854269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f6710 msgr2=0x55c6ca4095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f6710 0x55c6ca4095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 -- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 msgr2=0x55c6ca403470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 -- 172.21.15.49:0/181854269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ca195f70 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe8853e7640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe878005bc0 tx=0x7fe878005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee58245d0019e080 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878027070 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe878003800 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.009+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8780043d0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe878033050 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 10 client.15714 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c6ca02d170 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8780043d0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe878018070 con 0x55c6ca3f5a30 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 10 client.15714 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 20 client.15714 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca4130e0 0x55c6ca4154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 4 client.15714 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ca366610 con 0x55c6ca4130e0 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.010+0000 7fe884be6640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca4130e0 0x55c6ca4154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe884be6640 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca4130e0 0x55c6ca4154d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab8d68acf3a9606 server_cookie=eebb8dda7bd197ad in_seq=0 out_seq=0 2024-03-16T08:17:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe864ff9640 10 client.15714 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe864ff9640 1 -- 172.21.15.49:0/181854269 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c6ca366610 con 0x55c6ca4130e0 2024-03-16T08:17:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe864ff9640 10 client.15714 handle_command_reply: tid=0 2024-03-16T08:17:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 client.15714 shutdown 2024-03-16T08:17:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 msgr2=0x55c6ca409060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe878005bc0 tx=0x7fe878005d30 comp rx=0 tx=0).stop 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 shutdown_connections 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca4130e0 0x55c6ca4154d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f6710 0x55c6ca4095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe864ff9640 0 client.15714 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 --2- 172.21.15.49:0/181854269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409060 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.011+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 >> 172.21.15.49:0/181854269 conn(0x55c6ca3bcc50 msgr2=0x55c6ca3f21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.012+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 shutdown_connections 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.012+0000 7fe88a23b740 1 -- 172.21.15.49:0/181854269 wait complete. 2024-03-16T08:17:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.012+0000 7fe88a23b740 20 client.15714 trim_cache size 0 max 16384 2024-03-16T08:17:39.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.029+0000 7fe88a23b740 1 Processor -- start 2024-03-16T08:17:39.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.029+0000 7fe88a23b740 1 -- start start 2024-03-16T08:17:39.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801adab0 con 0x55c6ca3f6710 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801ab380 con 0x55c6ca3feda0 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe88a23b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801a91b0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51094/0 (socket says 172.21.15.49:51094) 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 -- 172.21.15.49:0/2118826158 learned_addr learned my addr 172.21.15.49:0/2118826158 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe885be8640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe8853e7640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 -- 172.21.15.49:0/2118826158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 msgr2=0x55c6ca409440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 -- 172.21.15.49:0/2118826158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 msgr2=0x55c6ca403020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 -- 172.21.15.49:0/2118826158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ca3f4a60 con 0x55c6ca3f6710 2024-03-16T08:17:39.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe8853e7640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe885be8640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe884be6640 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe87402cfd0 tx=0x7fe8740f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63b82c581229be3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe866ffd640 1 -- 172.21.15.49:0/2118826158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87404b040 con 0x55c6ca3f6710 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe866ffd640 1 -- 172.21.15.49:0/2118826158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8740bd530 con 0x55c6ca3f6710 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.030+0000 7fe866ffd640 1 -- 172.21.15.49:0/2118826158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740bb130 con 0x55c6ca3f6710 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- 172.21.15.49:0/2118826158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 msgr2=0x55c6ca3ef360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe87402cfd0 tx=0x7fe8740f2040 comp rx=0 tx=0).stop 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- 172.21.15.49:0/2118826158 shutdown_connections 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3feda0 0x55c6ca403020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca3ef360 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 --2- 172.21.15.49:0/2118826158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3f5a30 0x55c6ca409440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- 172.21.15.49:0/2118826158 >> 172.21.15.49:0/2118826158 conn(0x55c6ca3bcc50 msgr2=0x55c6ca3eec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- 172.21.15.49:0/2118826158 shutdown_connections 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- 172.21.15.49:0/2118826158 wait complete. 2024-03-16T08:17:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 -- start start 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.031+0000 7fe88a23b740 1 Processor -- start 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 0x55c6ca40cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3feda0 0x55c6ca478310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801a9840 con 0x55c6ca3f6710 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801aacb0 con 0x55c6ca3f5a30 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801a9f10 con 0x55c6ca3feda0 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe88a23b740 10 client.0 _ll_register_callbacks cb 0x7fe870011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51104/0 (socket says 172.21.15.49:51104) 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 -- 172.21.15.49:0/980915879 learned_addr learned my addr 172.21.15.49:0/980915879 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe8853e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 0x55c6ca40cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe885be8640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3feda0 0x55c6ca478310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 -- 172.21.15.49:0/980915879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3feda0 msgr2=0x55c6ca478310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3feda0 0x55c6ca478310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 -- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 msgr2=0x55c6ca40cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 0x55c6ca40cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 -- 172.21.15.49:0/980915879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ca478850 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe8853e7640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 0x55c6ca40cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe884be6640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe874009430 tx=0x7fe874009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830a657fb2f3e750 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.032+0000 7fe864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87404b070 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe8740b94a0 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8740bd530 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe8740bce30 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 10 client.15720 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c6ca479b00 con 0x55c6ca3f6710 2024-03-16T08:17:39.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8740bd530 con 0x55c6ca3f6710 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe8740ee170 con 0x55c6ca3f6710 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 10 client.15720 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 20 client.15720 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca47a250 0x55c6ca47c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 4 client.15720 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c6ca360960 con 0x55c6ca47a250 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.033+0000 7fe8853e7640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca47a250 0x55c6ca47c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.034+0000 7fe8853e7640 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca47a250 0x55c6ca47c670 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85759673c033cd2 server_cookie=f4b989272ed4a915 in_seq=0 out_seq=0 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.034+0000 7fe864ff9640 10 client.15720 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe864ff9640 1 -- 172.21.15.49:0/980915879 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c6ca360960 con 0x55c6ca47a250 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe864ff9640 10 client.15720 handle_command_reply: tid=0 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 client.15720 shutdown 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 msgr2=0x55c6ca40d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe874009430 tx=0x7fe874009260 comp rx=0 tx=0).stop 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 shutdown_connections 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c6ca47a250 0x55c6ca47c670 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6ca3feda0 0x55c6ca478310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c6ca3f6710 0x55c6ca40d2c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 --2- 172.21.15.49:0/980915879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6ca3f5a30 0x55c6ca40cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe864ff9640 0 client.15720 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 >> 172.21.15.49:0/980915879 conn(0x55c6ca3bcc50 msgr2=0x55c6ca4092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 shutdown_connections 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 1 -- 172.21.15.49:0/980915879 wait complete. 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.035+0000 7fe88a23b740 20 client.15720 trim_cache size 0 max 16384 2024-03-16T08:17:39.041 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:39.042 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:17:39.042 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "8d458022-1eab-4a9e-9fba-1f4d229476b3", 2024-03-16T08:17:39.042 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:17:39.042 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 msgr2=0x7fe8680a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 0x7fe8680a4ec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe870009b80 tx=0x7fe870009500 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 msgr2=0x7fe8801985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe874009230 tx=0x7fe87402cb30 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 shutdown_connections 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8680a2a00 0x7fe8680a4ec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe880106660 0x7fe88019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe880101010 0x7fe8801985e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 --2- 172.21.15.49:0/524053449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe880100650 0x7fe8801980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 >> 172.21.15.49:0/524053449 conn(0x7fe8800777e0 msgr2=0x7fe880068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 shutdown_connections 2024-03-16T08:17:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.036+0000 7fe88764b640 1 -- 172.21.15.49:0/524053449 wait complete. 2024-03-16T08:17:39.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:17:39.047 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 -- 172.21.15.49:0/2844022751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0106660 msgr2=0x7f80f0106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 --2- 172.21.15.49:0/2844022751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0106660 0x7f80f0106a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009860 tx=0x7f80e002bb70 comp rx=0 tx=0).stop 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 -- 172.21.15.49:0/2844022751 shutdown_connections 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 --2- 172.21.15.49:0/2844022751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 0x7f80f010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 --2- 172.21.15.49:0/2844022751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0100650 0x7f80f0100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 --2- 172.21.15.49:0/2844022751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0106660 0x7f80f0106a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 -- 172.21.15.49:0/2844022751 >> 172.21.15.49:0/2844022751 conn(0x7f80f00777e0 msgr2=0x7f80f0077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 -- 172.21.15.49:0/2844022751 shutdown_connections 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 -- 172.21.15.49:0/2844022751 wait complete. 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.157+0000 7f80f5d10640 1 Processor -- start 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 -- start start 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 0x7f80f0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 0x7f80f01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0110a70 con 0x7f80f0106660 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01108f0 con 0x7f80f0100650 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80f5d10640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0110bf0 con 0x7f80f0101010 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51124/0 (socket says 172.21.15.49:51124) 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 0x7f80f0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 -- 172.21.15.49:0/3935357838 learned_addr learned my addr 172.21.15.49:0/3935357838 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80eeffd640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 0x7f80f01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 -- 172.21.15.49:0/3935357838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 msgr2=0x7f80f01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 0x7f80f01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 -- 172.21.15.49:0/3935357838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 msgr2=0x7f80f0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 0x7f80f0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 -- 172.21.15.49:0/3935357838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f019d080 con 0x7f80f0106660 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 0x7f80f0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.158+0000 7f80effff640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80e800eb10 tx=0x7f80e800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.159+0000 7f80ecff9640 1 -- 172.21.15.49:0/3935357838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800ce20 con 0x7f80f0106660 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.159+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f019d370 con 0x7f80f0106660 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.159+0000 7f80ecff9640 1 -- 172.21.15.49:0/3935357838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e8005020 con 0x7f80f0106660 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.159+0000 7f80ecff9640 1 -- 172.21.15.49:0/3935357838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8010690 con 0x7f80f0106660 2024-03-16T08:17:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.159+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01a4c00 con 0x7f80f0106660 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80ecff9640 1 -- 172.21.15.49:0/3935357838 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f80e8005540 con 0x7f80f0106660 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 Processor -- start 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 -- start start 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80ecff9640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 0x7f80d80a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ee9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560a48e42500 con 0x560a48ede150 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a48deabe0 con 0x560a48ee7650 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80f89de740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a48e3e5a0 con 0x560a48edea70 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80eeffd640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 0x7f80d80a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.161+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51130/0 (socket says 172.21.15.49:51130) 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 -- 172.21.15.49:0/3274825350 learned_addr learned my addr 172.21.15.49:0/3274825350 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ecff9640 1 -- 172.21.15.49:0/3935357838 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f80e80cf5f0 con 0x7f80f0106660 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80effff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ee9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80eeffd640 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 -- 172.21.15.49:0/3274825350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 msgr2=0x560a48ee7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 -- 172.21.15.49:0/3274825350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 msgr2=0x560a48ee9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ee9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 -- 172.21.15.49:0/3274825350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a48dfe4b0 con 0x560a48ede150 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80effff640 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ee9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80eeffd640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 0x7f80d80a4ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80f00ffa10 tx=0x7f80e4009b10 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f80e002c0b0 tx=0x7f80e00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c3918434171e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80d5ffb640 1 -- 172.21.15.49:0/3274825350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0039040 con 0x560a48ede150 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80d5ffb640 1 -- 172.21.15.49:0/3274825350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0004230 con 0x560a48ede150 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.162+0000 7f80d5ffb640 1 -- 172.21.15.49:0/3274825350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e002c910 con 0x560a48ede150 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- 172.21.15.49:0/3274825350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 msgr2=0x560a48ede530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f80e002c0b0 tx=0x7f80e00058d0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- 172.21.15.49:0/3274825350 shutdown_connections 2024-03-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ee9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 --2- 172.21.15.49:0/3274825350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ede530 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- 172.21.15.49:0/3274825350 >> 172.21.15.49:0/3274825350 conn(0x560a48ed8880 msgr2=0x560a48ed8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- 172.21.15.49:0/3274825350 shutdown_connections 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- 172.21.15.49:0/3274825350 wait complete. 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.163+0000 7f80f89de740 1 -- start start 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 Processor -- start 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 0x560a48ee4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48ee7650 0x560a48ee50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560a48eea3e0 con 0x560a48edea70 2024-03-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a48dfe160 con 0x560a48ede150 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a48eea560 con 0x560a48ee7650 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80f89de740 10 client.0 _ll_register_callbacks cb 0x560a48de2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51136/0 (socket says 172.21.15.49:51136) 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 -- 172.21.15.49:0/350381798 learned_addr learned my addr 172.21.15.49:0/350381798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 0x560a48ee4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80effff640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48ee7650 0x560a48ee50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 -- 172.21.15.49:0/350381798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48ee7650 msgr2=0x560a48ee50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48ee7650 0x560a48ee50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 -- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 msgr2=0x560a48ee4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 0x560a48ee4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 -- 172.21.15.49:0/350381798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a48c207d0 con 0x560a48edea70 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 0x560a48ee4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.164+0000 7f80eeffd640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80e40132a0 tx=0x7f80e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a10623f2b3085 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4005020 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f80e40051c0 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0004230 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80e40053c0 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 10 client.15732 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560a48b563f0 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0004230 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80e4020050 con 0x560a48edea70 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 10 client.15732 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 20 client.15732 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:39.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48ef74d0 0x560a48ef9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 4 client.15732 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.165+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a48e4fa40 con 0x560a48ef74d0 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48ef74d0 0x560a48ef9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48ef74d0 0x560a48ef9920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795b08e029dd371f server_cookie=184d5c269c85349 in_seq=0 out_seq=0 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80ceffd640 10 client.15732 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80ceffd640 1 -- 172.21.15.49:0/350381798 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560a48e4fa40 con 0x560a48ef74d0 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80ceffd640 10 client.15732 handle_command_reply: tid=0 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.166+0000 7f80f89de740 1 client.15732 shutdown 2024-03-16T08:17:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 msgr2=0x560a48ee4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80e40132a0 tx=0x7f80e4004730 comp rx=0 tx=0).stop 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 shutdown_connections 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48ef74d0 0x560a48ef9920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48ee7650 0x560a48ee50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48edea70 0x560a48ee4b60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80ceffd640 0 client.15732 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 --2- 172.21.15.49:0/350381798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ede150 0x560a48ee4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 >> 172.21.15.49:0/350381798 conn(0x560a48ed8880 msgr2=0x560a48eda5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 shutdown_connections 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 1 -- 172.21.15.49:0/350381798 wait complete. 2024-03-16T08:17:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.167+0000 7f80f89de740 20 client.15732 trim_cache size 0 max 16384 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 Processor -- start 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 -- start start 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ef3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01afcc0 con 0x560a48ede150 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ad590 con 0x560a48ee7650 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80f89de740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ab3c0 con 0x560a48edea70 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51144/0 (socket says 172.21.15.49:51144) 2024-03-16T08:17:39.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.184+0000 7f80ef7fe640 1 -- 172.21.15.49:0/1745929606 learned_addr learned my addr 172.21.15.49:0/1745929606 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80eeffd640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80effff640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ef3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 -- 172.21.15.49:0/1745929606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 msgr2=0x560a48ee3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 -- 172.21.15.49:0/1745929606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 msgr2=0x560a48ef3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ef3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 -- 172.21.15.49:0/1745929606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a48b160f0 con 0x560a48ede150 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80effff640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ef3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009860 tx=0x7f80e0041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3141c836e6b01ed server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80d57fa640 1 -- 172.21.15.49:0/1745929606 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0039040 con 0x560a48ede150 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80d57fa640 1 -- 172.21.15.49:0/1745929606 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0032340 con 0x560a48ede150 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.185+0000 7f80d57fa640 1 -- 172.21.15.49:0/1745929606 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e002c510 con 0x560a48ede150 2024-03-16T08:17:39.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- 172.21.15.49:0/1745929606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 msgr2=0x560a48ee6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009860 tx=0x7f80e0041040 comp rx=0 tx=0).stop 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- 172.21.15.49:0/1745929606 shutdown_connections 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48ef3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48ee3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- 172.21.15.49:0/1745929606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48ee6d50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- 172.21.15.49:0/1745929606 >> 172.21.15.49:0/1745929606 conn(0x560a48ed8880 msgr2=0x560a48eda520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- 172.21.15.49:0/1745929606 shutdown_connections 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- 172.21.15.49:0/1745929606 wait complete. 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 -- start start 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 Processor -- start 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48eeb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.186+0000 7f80f89de740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48f5c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01aba50 con 0x560a48ede150 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01acec0 con 0x560a48ee7650 2024-03-16T08:17:39.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ac120 con 0x560a48edea70 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 10 client.0 _ll_register_callbacks cb 0x7f80e4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80eeffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48eeb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51152/0 (socket says 172.21.15.49:51152) 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 -- 172.21.15.49:0/658064914 learned_addr learned my addr 172.21.15.49:0/658064914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80effff640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48f5c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 -- 172.21.15.49:0/658064914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 msgr2=0x560a48eeb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48eeb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 -- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 msgr2=0x560a48f5c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48f5c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 -- 172.21.15.49:0/658064914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a48e099d0 con 0x560a48ede150 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ef7fe640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f80e0050560 tx=0x7f80e002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ee16efce91d3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e005f060 con 0x560a48ede150 2024-03-16T08:17:39.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f80e0034750 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0032340 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80e0039040 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 10 client.15738 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.187+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560a48f5dce0 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0032340 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80e00508e0 con 0x560a48ede150 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 10 client.15738 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 20 client.15738 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48f5e010 0x560a48f60400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 4 client.15738 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a48e49d90 con 0x560a48f5e010 2024-03-16T08:17:39.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80eeffd640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48f5e010 0x560a48f60400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80ceffd640 10 client.15738 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80eeffd640 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48f5e010 0x560a48f60400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3db99497cee4523 server_cookie=88672e22ba7154ee in_seq=0 out_seq=0 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80ceffd640 1 -- 172.21.15.49:0/658064914 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a48e49d90 con 0x560a48f5e010 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80ceffd640 10 client.15738 handle_command_reply: tid=0 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.188+0000 7f80f89de740 1 client.15738 shutdown 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 msgr2=0x560a48eeb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f80e0050560 tx=0x7f80e002bb70 comp rx=0 tx=0).stop 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 shutdown_connections 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560a48f5e010 0x560a48f60400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a48ee7650 0x560a48f5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80ceffd640 0 client.15738 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a48edea70 0x560a48eeb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 --2- 172.21.15.49:0/658064914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a48ede150 0x560a48eeb430 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 >> 172.21.15.49:0/658064914 conn(0x560a48ed8880 msgr2=0x560a48ee45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 shutdown_connections 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 1 -- 172.21.15.49:0/658064914 wait complete. 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f89de740 20 client.15738 trim_cache size 0 max 16384 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "8d458022-1eab-4a9e-9fba-1f4d229476b3": { 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "8d458022-1eab-4a9e-9fba-1f4d229476b3", 2024-03-16T08:17:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:17:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:17:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:17:39.197 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 msgr2=0x7f80d80a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 0x7f80d80a4ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80f00ffa10 tx=0x7f80e4009b10 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 msgr2=0x7f80f019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80e800eb10 tx=0x7f80e800ef10 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 shutdown_connections 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80d80a2a00 0x7f80d80a4ec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f0106660 0x7f80f019c950 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80f0101010 0x7f80f01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 --2- 172.21.15.49:0/3935357838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80f0100650 0x7f80f0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.189+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 >> 172.21.15.49:0/3935357838 conn(0x7f80f00777e0 msgr2=0x7f80f0068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.190+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 shutdown_connections 2024-03-16T08:17:39.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:17:39.190+0000 7f80f5d10640 1 -- 172.21.15.49:0/3935357838 wait complete. 2024-03-16T08:17:39.200 INFO:tasks.cephfs.filesystem:scrub status for tag:8d458022-1eab-4a9e-9fba-1f4d229476b3 - {'path': '/', 'tag': '8d458022-1eab-4a9e-9fba-1f4d229476b3', 'options': 'recursive,force'} 2024-03-16T08:17:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:40 smithi154 ceph-mon[36471]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-16T08:17:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:40 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:17:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:40 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:40.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:40 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:17:40.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:40 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:40 smithi049 ceph-mon[31942]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-16T08:17:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:40 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:17:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:40 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:40 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:17:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:40 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:40 smithi129 ceph-mon[36561]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-16T08:17:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:40 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:17:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:40 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:40 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:17:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:40 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:17:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 -- 172.21.15.49:0/2356098703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c1012f0 msgr2=0x7f9d8c10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 --2- 172.21.15.49:0/2356098703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c10d510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009900 tx=0x7f9d8402bc30 comp rx=0 tx=0).stop 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 -- 172.21.15.49:0/2356098703 shutdown_connections 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 --2- 172.21.15.49:0/2356098703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c10d510 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 --2- 172.21.15.49:0/2356098703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100930 0x7f9d8c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 --2- 172.21.15.49:0/2356098703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c105680 0x7f9d8c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 -- 172.21.15.49:0/2356098703 >> 172.21.15.49:0/2356098703 conn(0x7f9d8c077530 msgr2=0x7f9d8c077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 -- 172.21.15.49:0/2356098703 shutdown_connections 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 -- 172.21.15.49:0/2356098703 wait complete. 2024-03-16T08:17:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.342+0000 7f9d91acc640 1 Processor -- start 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 -- start start 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 0x7f9d8c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c110b60 con 0x7f9d8c105680 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1109e0 con 0x7f9d8c100930 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d91acc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c110ce0 con 0x7f9d8c1012f0 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 0x7f9d8c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51158/0 (socket says 172.21.15.49:51158) 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8affd640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 -- 172.21.15.49:0/626971504 learned_addr learned my addr 172.21.15.49:0/626971504 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 -- 172.21.15.49:0/626971504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 msgr2=0x7f9d8c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 -- 172.21.15.49:0/626971504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 msgr2=0x7f9d8c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 0x7f9d8c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 -- 172.21.15.49:0/626971504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19cfe0 con 0x7f9d8c105680 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8affd640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 0x7f9d8c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.343+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402efd0 tx=0x7f9d840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.344+0000 7f9d88ff9640 1 -- 172.21.15.49:0/626971504 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84041070 con 0x7f9d8c105680 2024-03-16T08:17:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.344+0000 7f9d88ff9640 1 -- 172.21.15.49:0/626971504 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8402ccc0 con 0x7f9d8c105680 2024-03-16T08:17:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.344+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c19d270 con 0x7f9d8c105680 2024-03-16T08:17:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.344+0000 7f9d88ff9640 1 -- 172.21.15.49:0/626971504 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84003b30 con 0x7f9d8c105680 2024-03-16T08:17:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.344+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a4b60 con 0x7f9d8c105680 2024-03-16T08:17:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.346+0000 7f9d88ff9640 1 -- 172.21.15.49:0/626971504 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x7f9d8c105680 2024-03-16T08:17:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.346+0000 7f9d9478a740 1 Processor -- start 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.346+0000 7f9d9478a740 1 -- start start 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d88ff9640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 0x7f9d780a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce21812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2181820 0x563ce2183c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ce20dca40 con 0x563ce2178a50 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563ce2085920 con 0x563ce2181820 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d9478a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ce20d8ae0 con 0x563ce2178130 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8affd640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 0x7f9d780a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce21812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:52190/0 (socket says 172.21.15.49:52190) 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/3301412849 learned_addr learned my addr 172.21.15.49:0/3301412849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2181820 0x563ce2183c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d88ff9640 1 -- 172.21.15.49:0/626971504 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9d840f0ac0 con 0x7f9d8c105680 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/3301412849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2181820 msgr2=0x563ce2183c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2181820 0x563ce2183c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/3301412849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 msgr2=0x563ce21812e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce21812e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/3301412849 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce209d460 con 0x563ce2178130 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.347+0000 7f9d8affd640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 0x7f9d780a4ff0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009730 tx=0x7f9d74009b50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c011860 tx=0x7f9d7c011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbbbab3bece2119 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d81ffb640 1 -- 172.21.15.49:0/3301412849 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0112f0 con 0x563ce2178130 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d81ffb640 1 -- 172.21.15.49:0/3301412849 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c00f9a0 con 0x563ce2178130 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d81ffb640 1 -- 172.21.15.49:0/3301412849 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c005700 con 0x563ce2178130 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 -- 172.21.15.49:0/3301412849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 msgr2=0x563ce2178510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c011860 tx=0x7f9d7c011d20 comp rx=0 tx=0).stop 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 -- 172.21.15.49:0/3301412849 shutdown_connections 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2181820 0x563ce2183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce21812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 --2- 172.21.15.49:0/3301412849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2178510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.348+0000 7f9d9478a740 1 -- 172.21.15.49:0/3301412849 >> 172.21.15.49:0/3301412849 conn(0x563ce2172a50 msgr2=0x563ce2172e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 -- 172.21.15.49:0/3301412849 shutdown_connections 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 -- 172.21.15.49:0/3301412849 wait complete. 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 -- start start 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 Processor -- start 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce217ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce217f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ce2184450 con 0x563ce2178130 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.349+0000 7f9d9478a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563ce2098d70 con 0x563ce2178a50 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d9478a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ce21845d0 con 0x563ce2181820 2024-03-16T08:17:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d9478a740 10 client.0 _ll_register_callbacks cb 0x563ce1df0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51178/0 (socket says 172.21.15.49:51178) 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/2530230366 learned_addr learned my addr 172.21.15.49:0/2530230366 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8affd640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce217ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce217f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/2530230366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 msgr2=0x563ce217f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce217f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 msgr2=0x563ce217ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce217ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/2530230366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce1df1210 con 0x563ce2178130 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8affd640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce217ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009b60 tx=0x7f9d7c011320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1698d0ab3af46ce8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c028030 con 0x563ce2178130 2024-03-16T08:17:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9d7c004c50 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.350+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c00f9a0 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c032050 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 10 client.15750 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ce1ebd620 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c00f9a0 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9d7c018070 con 0x563ce2178130 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 10 client.15750 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 20 client.15750 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce2191920 0x563ce2193d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 4 client.15750 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce20e9f80 con 0x563ce2191920 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.351+0000 7f9d8affd640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce2191920 0x563ce2193d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.352+0000 7f9d8affd640 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce2191920 0x563ce2193d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3def232f499e7 server_cookie=56b2c4868a3f01e8 in_seq=0 out_seq=0 2024-03-16T08:17:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.352+0000 7f9d6affd640 10 client.15750 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d6affd640 1 -- 172.21.15.49:0/2530230366 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563ce20e9f80 con 0x563ce2191920 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d6affd640 10 client.15750 handle_command_reply: tid=0 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 client.15750 shutdown 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 msgr2=0x563ce217e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009b60 tx=0x7f9d7c011320 comp rx=0 tx=0).stop 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 shutdown_connections 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce2191920 0x563ce2193d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce217f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce217ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 --2- 172.21.15.49:0/2530230366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178130 0x563ce217e840 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d6affd640 0 client.15750 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 >> 172.21.15.49:0/2530230366 conn(0x563ce2172a50 msgr2=0x563ce21747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 shutdown_connections 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 1 -- 172.21.15.49:0/2530230366 wait complete. 2024-03-16T08:17:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.353+0000 7f9d9478a740 20 client.15750 trim_cache size 0 max 16384 2024-03-16T08:17:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 Processor -- start 2024-03-16T08:17:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 -- start start 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 0x563ce2180f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce216da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1afbb0 con 0x563ce2178a50 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ad480 con 0x563ce2178130 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d9478a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ab2d0 con 0x563ce2181820 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d8affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 0x563ce2180f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.368+0000 7f9d8affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51180/0 (socket says 172.21.15.49:51180) 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce216da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8affd640 1 -- 172.21.15.49:0/1325917188 learned_addr learned my addr 172.21.15.49:0/1325917188 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/1325917188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 msgr2=0x563ce216da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce216da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/1325917188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 msgr2=0x563ce218d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 -- 172.21.15.49:0/1325917188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce217d7f0 con 0x563ce2178130 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8affd640 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 0x563ce2180f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005700 tx=0x7f9d7c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821021cad9f6b4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d817fa640 1 -- 172.21.15.49:0/1325917188 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c028040 con 0x563ce2178130 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d817fa640 1 -- 172.21.15.49:0/1325917188 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c01f040 con 0x563ce2178130 2024-03-16T08:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.369+0000 7f9d817fa640 1 -- 172.21.15.49:0/1325917188 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c004280 con 0x563ce2178130 2024-03-16T08:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- 172.21.15.49:0/1325917188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 msgr2=0x563ce2180f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 0x563ce2180f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005700 tx=0x7f9d7c009650 comp rx=0 tx=0).stop 2024-03-16T08:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- 172.21.15.49:0/1325917188 shutdown_connections 2024-03-16T08:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2181820 0x563ce216da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2178a50 0x563ce218d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1325917188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178130 0x563ce2180f70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- 172.21.15.49:0/1325917188 >> 172.21.15.49:0/1325917188 conn(0x563ce2172a50 msgr2=0x563ce2171bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- 172.21.15.49:0/1325917188 shutdown_connections 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- 172.21.15.49:0/1325917188 wait complete. 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 -- start start 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.370+0000 7f9d9478a740 1 Processor -- start 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2185020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce2185560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ab940 con 0x563ce2181820 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1acdb0 con 0x563ce2178a50 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ac010 con 0x563ce2178130 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d9478a740 10 client.0 _ll_register_callbacks cb 0x7f9d74010730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51196/0 (socket says 172.21.15.49:51196) 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 -- 172.21.15.49:0/1617989749 learned_addr learned my addr 172.21.15.49:0/1617989749 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2185020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8affd640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce2185560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 -- 172.21.15.49:0/1617989749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 msgr2=0x563ce2185020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2185020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 -- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 msgr2=0x563ce2185560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce2185560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 -- 172.21.15.49:0/1617989749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce21f6b70 con 0x563ce2181820 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d8bfff640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b9620 tx=0x7f9d8402ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e435c8847d2b8006 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.371+0000 7f9d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840bd1c0 con 0x563ce2181820 2024-03-16T08:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9d840b9830 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c01f040 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d840c1260 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 10 client.15756 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ce21f7de0 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c01f040 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9d840bd420 con 0x563ce2181820 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 10 client.15756 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 20 client.15756 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce21f8500 0x563ce21fa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 4 client.15756 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ce20e42d0 con 0x563ce21f8500 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.372+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce21f8500 0x563ce21fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.373+0000 7f9d6affd640 10 client.15756 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.373+0000 7f9d8b7fe640 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce21f8500 0x563ce21fa920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b25365443914c4 server_cookie=9f9685efa80ef69e in_seq=0 out_seq=0 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d6affd640 1 -- 172.21.15.49:0/1617989749 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ce20e42d0 con 0x563ce21f8500 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d6affd640 10 client.15756 handle_command_reply: tid=0 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 client.15756 shutdown 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 msgr2=0x563ce2185aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b9620 tx=0x7f9d8402ce70 comp rx=0 tx=0).stop 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 shutdown_connections 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563ce21f8500 0x563ce21fa920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ce2181820 0x563ce2185aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ce2178a50 0x563ce2185560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d6affd640 0 client.15756 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 --2- 172.21.15.49:0/1617989749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce2178130 0x563ce2185020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 >> 172.21.15.49:0/1617989749 conn(0x563ce2172a50 msgr2=0x563ce217e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 shutdown_connections 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 1 -- 172.21.15.49:0/1617989749 wait complete. 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.376+0000 7f9d9478a740 20 client.15756 trim_cache size 0 max 16384 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 msgr2=0x7f9d780a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 0x7f9d780a4ff0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009730 tx=0x7f9d74009b50 comp rx=0 tx=0).stop 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 msgr2=0x7f9d8c19c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402efd0 tx=0x7f9d840042f0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 shutdown_connections 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d780a2b30 0x7f9d780a4ff0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d8c105680 0x7f9d8c19c8b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c1012f0 0x7f9d8c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 --2- 172.21.15.49:0/626971504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d8c100930 0x7f9d8c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 >> 172.21.15.49:0/626971504 conn(0x7f9d8c077530 msgr2=0x7f9d8c0690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 shutdown_connections 2024-03-16T08:17:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:41.377+0000 7f9d91acc640 1 -- 172.21.15.49:0/626971504 wait complete. 2024-03-16T08:17:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:42 smithi154 ceph-mon[36471]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:17:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:42 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:17:42.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:42 smithi049 ceph-mon[31942]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:17:42.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:42 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:17:42.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:42 smithi129 ceph-mon[36561]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T08:17:42.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:42 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:17:44.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:44 smithi154 ceph-mon[36471]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:17:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:44 smithi049 ceph-mon[31942]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:17:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:44 smithi129 ceph-mon[36561]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:17:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:46 smithi154 ceph-mon[36471]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:17:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:46 smithi049 ceph-mon[31942]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:17:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:46 smithi129 ceph-mon[36561]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:17:48.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:48 smithi049 ceph-mon[31942]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:17:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:48 smithi129 ceph-mon[36561]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:17:48.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:48 smithi154 ceph-mon[36471]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:17:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:50 smithi049 ceph-mon[31942]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-16T08:17:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:50 smithi129 ceph-mon[36561]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-16T08:17:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:50 smithi154 ceph-mon[36471]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-16T08:17:52.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:52 smithi049 ceph-mon[31942]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:52.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:52 smithi129 ceph-mon[36561]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:52 smithi154 ceph-mon[36471]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 -- 172.21.15.49:0/2057802480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80201012f0 msgr2=0x7f802010d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 --2- 172.21.15.49:0/2057802480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80201012f0 0x7f802010d510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8018009ca0 tx=0x7f801802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 -- 172.21.15.49:0/2057802480 shutdown_connections 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 --2- 172.21.15.49:0/2057802480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80201012f0 0x7f802010d510 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 --2- 172.21.15.49:0/2057802480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8020100930 0x7f8020100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 --2- 172.21.15.49:0/2057802480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 -- 172.21.15.49:0/2057802480 >> 172.21.15.49:0/2057802480 conn(0x7f8020077530 msgr2=0x7f8020077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 -- 172.21.15.49:0/2057802480 shutdown_connections 2024-03-16T08:17:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.466+0000 7f802480b640 1 -- 172.21.15.49:0/2057802480 wait complete. 2024-03-16T08:17:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 Processor -- start 2024-03-16T08:17:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 -- start start 2024-03-16T08:17:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 0x7f8020106a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8020110ba0 con 0x7f8020100930 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8020110a20 con 0x7f8020105680 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f802480b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8020110d20 con 0x7f80201012f0 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40692/0 (socket says 172.21.15.49:40692) 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801e59c640 1 -- 172.21.15.49:0/2868316407 learned_addr learned my addr 172.21.15.49:0/2868316407 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 0x7f8020106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801ed9d640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 -- 172.21.15.49:0/2868316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 msgr2=0x7f8020106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 -- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 msgr2=0x7f80201064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.467+0000 7f801dd9b640 1 -- 172.21.15.49:0/2868316407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80201a6580 con 0x7f80201012f0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f801e59c640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f801ed9d640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 0x7f8020106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9f0 tx=0x7f801400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f800f7fe640 1 -- 172.21.15.49:0/2868316407 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400ccc0 con 0x7f80201012f0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80201a6870 con 0x7f80201012f0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f800f7fe640 1 -- 172.21.15.49:0/2868316407 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014010910 con 0x7f80201012f0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f800f7fe640 1 -- 172.21.15.49:0/2868316407 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014018960 con 0x7f80201012f0 2024-03-16T08:17:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.468+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a6e10 con 0x7f80201012f0 2024-03-16T08:17:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.470+0000 7f800f7fe640 1 -- 172.21.15.49:0/2868316407 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f8014010ac0 con 0x7f80201012f0 2024-03-16T08:17:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.471+0000 7f800f7fe640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 0x7f80080a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.471+0000 7f801e59c640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 0x7f80080a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.471+0000 7f800f7fe640 1 -- 172.21.15.49:0/2868316407 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f80140cf370 con 0x7f80201012f0 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.471+0000 7f801e59c640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 0x7f80080a4f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8010011440 tx=0x7f8010005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.471+0000 7f80273f5740 1 Processor -- start 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 -- start start 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 0x55c41b6972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b699c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c41b5f2a40 con 0x55c41b68e130 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c41b59b920 con 0x55c41b68ea50 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f80273f5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c41b5eeae0 con 0x55c41b697820 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40696/0 (socket says 172.21.15.49:40696) 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801e59c640 1 -- 172.21.15.49:0/3641134173 learned_addr learned my addr 172.21.15.49:0/3641134173 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 0x55c41b6972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801ed9d640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b699c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 -- 172.21.15.49:0/3641134173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 msgr2=0x55c41b699c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b699c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 -- 172.21.15.49:0/3641134173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 msgr2=0x55c41b68e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801dd9b640 1 -- 172.21.15.49:0/3641134173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41b5b3460 con 0x55c41b68ea50 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801e59c640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.472+0000 7f801ed9d640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b699c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.473+0000 7f801dd9b640 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 0x55c41b6972e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80140cca50 tx=0x7f801409b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d7a1b3e213d4bde server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.473+0000 7f800cff9640 1 -- 172.21.15.49:0/3641134173 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140a4050 con 0x55c41b68ea50 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.473+0000 7f800cff9640 1 -- 172.21.15.49:0/3641134173 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014097fd0 con 0x55c41b68ea50 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.473+0000 7f800cff9640 1 -- 172.21.15.49:0/3641134173 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140862d0 con 0x55c41b68ea50 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- 172.21.15.49:0/3641134173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 msgr2=0x55c41b6972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 0x55c41b6972e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80140cca50 tx=0x7f801409b960 comp rx=0 tx=0).stop 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- 172.21.15.49:0/3641134173 shutdown_connections 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b699c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68ea50 0x55c41b6972e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 --2- 172.21.15.49:0/3641134173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b68e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- 172.21.15.49:0/3641134173 >> 172.21.15.49:0/3641134173 conn(0x55c41b688a50 msgr2=0x55c41b688e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- 172.21.15.49:0/3641134173 shutdown_connections 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- 172.21.15.49:0/3641134173 wait complete. 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 -- start start 2024-03-16T08:17:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.474+0000 7f80273f5740 1 Processor -- start 2024-03-16T08:17:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 0x55c41b694940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b6953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c41b69a3f0 con 0x55c41b68ea50 2024-03-16T08:17:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c41b5aed70 con 0x55c41b68e130 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c41b69a570 con 0x55c41b697820 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f80273f5740 10 client.0 _ll_register_callbacks cb 0x55c41b306a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40712/0 (socket says 172.21.15.49:40712) 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 -- 172.21.15.49:0/2934614458 learned_addr learned my addr 172.21.15.49:0/2934614458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801e59c640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 0x55c41b694940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801ed9d640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b6953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 -- 172.21.15.49:0/2934614458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 msgr2=0x55c41b6953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b6953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 -- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 msgr2=0x55c41b694940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 0x55c41b694940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 -- 172.21.15.49:0/2934614458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41b307210 con 0x55c41b68ea50 2024-03-16T08:17:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801e59c640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 0x55c41b694940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f801dd9b640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f801400c6a0 tx=0x7f801409ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9256a9cda5f39f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.475+0000 7f8005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140b9070 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f80140862d0 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8014097fd0 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80140a4050 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c41b3d3620 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014097fd0 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80140a6080 con 0x55c41b68ea50 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 20 client.15768 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b6a78c0 0x55c41b6a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 4 client.15768 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.476+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c41b5fff80 con 0x55c41b6a78c0 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.477+0000 7f801e59c640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b6a78c0 0x55c41b6a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.477+0000 7f801e59c640 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b6a78c0 0x55c41b6a9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6591c20ef4c3a2 server_cookie=b23d8a8e03b37b11 in_seq=0 out_seq=0 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.477+0000 7f8005ffb640 10 client.15768 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f8005ffb640 1 -- 172.21.15.49:0/2934614458 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c41b5fff80 con 0x55c41b6a78c0 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f8005ffb640 10 client.15768 handle_command_reply: tid=0 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 client.15768 shutdown 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 msgr2=0x55c41b694e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f801400c6a0 tx=0x7f801409ce10 comp rx=0 tx=0).stop 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 shutdown_connections 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b6a78c0 0x55c41b6a9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b697820 0x55c41b6953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68ea50 0x55c41b694e80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f8005ffb640 0 client.15768 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 --2- 172.21.15.49:0/2934614458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b68e130 0x55c41b694940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 >> 172.21.15.49:0/2934614458 conn(0x55c41b688a50 msgr2=0x55c41b68a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 shutdown_connections 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 1 -- 172.21.15.49:0/2934614458 wait complete. 2024-03-16T08:17:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.478+0000 7f80273f5740 20 client.15768 trim_cache size 0 max 16384 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 Processor -- start 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 -- start start 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b696060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b1e40 con 0x55c41b68e130 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80201af710 con 0x55c41b697820 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f80273f5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80201ad540 con 0x55c41b68ea50 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40716/0 (socket says 172.21.15.49:40716) 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f801e59c640 1 -- 172.21.15.49:0/1645091778 learned_addr learned my addr 172.21.15.49:0/1645091778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f801dd9b640 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.493+0000 7f801ed9d640 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b696060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 -- 172.21.15.49:0/1645091778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 msgr2=0x55c41b6920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 -- 172.21.15.49:0/1645091778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 msgr2=0x55c41b696060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b696060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 -- 172.21.15.49:0/1645091778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41b692e90 con 0x55c41b68e130 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f801e59c640 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8010005d00 tx=0x7f8010014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce18a0f82d99f52 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f8007fff640 1 -- 172.21.15.49:0/1645091778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010005220 con 0x55c41b68e130 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f8007fff640 1 -- 172.21.15.49:0/1645091778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010004c80 con 0x55c41b68e130 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.494+0000 7f8007fff640 1 -- 172.21.15.49:0/1645091778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801002c5d0 con 0x55c41b68e130 2024-03-16T08:17:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- 172.21.15.49:0/1645091778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 msgr2=0x55c41b691ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8010005d00 tx=0x7f8010014000 comp rx=0 tx=0).stop 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- 172.21.15.49:0/1645091778 shutdown_connections 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b696060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- 172.21.15.49:0/1645091778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b691ba0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- 172.21.15.49:0/1645091778 >> 172.21.15.49:0/1645091778 conn(0x55c41b688a50 msgr2=0x55c41b68fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- 172.21.15.49:0/1645091778 shutdown_connections 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- 172.21.15.49:0/1645091778 wait complete. 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- start start 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 Processor -- start 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b70c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80201adbd0 con 0x55c41b68e130 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80201af040 con 0x55c41b697820 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80201ae2a0 con 0x55c41b68ea50 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f80273f5740 10 client.0 _ll_register_callbacks cb 0x7f8010009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.495+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40724/0 (socket says 172.21.15.49:40724) 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 -- 172.21.15.49:0/849957407 learned_addr learned my addr 172.21.15.49:0/849957407 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801dd9b640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801ed9d640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b70c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 -- 172.21.15.49:0/849957407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 msgr2=0x55c41b6aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 -- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 msgr2=0x55c41b70c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b70c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 -- 172.21.15.49:0/849957407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41b70cda0 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801dd9b640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f801e59c640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8010005f50 tx=0x7f801000feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8b38447c0c128e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f8005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010004880 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8010004c80 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8010004a20 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f801002c310 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f80273f5740 10 client.15774 fetch_fsmap learned FSMap version 24 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.496+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c41b70df10 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010004c80 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8010029450 con 0x55c41b68e130 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 10 client.15774 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:17:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 20 client.15774 populate_metadata read hostname 'smithi049' 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b70e6e0 0x55c41b710b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 4 client.15774 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c41b5fa2d0 con 0x55c41b70e6e0 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f801dd9b640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b70e6e0 0x55c41b710b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f8005ffb640 10 client.15774 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.497+0000 7f801dd9b640 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b70e6e0 0x55c41b710b00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695dee71062c73fd server_cookie=4507a5d6de13bf5 in_seq=0 out_seq=0 2024-03-16T08:17:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.503+0000 7f8005ffb640 1 -- 172.21.15.49:0/849957407 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c41b5fa2d0 con 0x55c41b70e6e0 2024-03-16T08:17:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f8005ffb640 10 client.15774 handle_command_reply: tid=0 2024-03-16T08:17:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 client.15774 shutdown 2024-03-16T08:17:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 msgr2=0x55c41b6aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8010005f50 tx=0x7f801000feb0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 shutdown_connections 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c41b70e6e0 0x55c41b710b00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c41b697820 0x55c41b70c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c41b68ea50 0x55c41b6aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 --2- 172.21.15.49:0/849957407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c41b68e130 0x55c41b6aa9b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f8005ffb640 0 client.15774 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 >> 172.21.15.49:0/849957407 conn(0x55c41b688a50 msgr2=0x55c41b6a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 shutdown_connections 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 1 -- 172.21.15.49:0/849957407 wait complete. 2024-03-16T08:17:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f80273f5740 20 client.15774 trim_cache size 0 max 16384 2024-03-16T08:17:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:17:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:17:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:17:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 msgr2=0x7f80080a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 0x7f80080a4f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8010011440 tx=0x7f8010005f90 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 msgr2=0x7f8020106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.504+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 0x7f8020106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9f0 tx=0x7f801400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 shutdown_connections 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f80080a2ab0 0x7f80080a4f70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8020105680 0x7f8020106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80201012f0 0x7f8020106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 --2- 172.21.15.49:0/2868316407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8020100930 0x7f80201064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:17:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 >> 172.21.15.49:0/2868316407 conn(0x7f8020077530 msgr2=0x7f8020068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:17:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 shutdown_connections 2024-03-16T08:17:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:17:54.505+0000 7f802480b640 1 -- 172.21.15.49:0/2868316407 wait complete. 2024-03-16T08:17:54.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:54 smithi049 ceph-mon[31942]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:54.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:54 smithi129 ceph-mon[36561]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:54.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:54 smithi154 ceph-mon[36471]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-16T08:17:56.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:56 smithi049 ceph-mon[31942]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-16T08:17:56.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:56 smithi129 ceph-mon[36561]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-16T08:17:56.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:56 smithi154 ceph-mon[36471]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-16T08:17:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:17:58 smithi049 ceph-mon[31942]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-16T08:17:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:17:58 smithi129 ceph-mon[36561]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-16T08:17:58.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:17:58 smithi154 ceph-mon[36471]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-16T08:18:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-mon[31942]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T08:18:00.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:18:00.237+0000 7fa8177e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:18:00.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:18:00.237+0000 7fa8177e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:18:00.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:18:00.237+0000 7fa8177e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:18:00.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:18:00.237+0000 7fa8177e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:18:00.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:18:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:18:00.237+0000 7fa8177e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T08:18:00.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:00 smithi129 ceph-mon[36561]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T08:18:00.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:00 smithi154 ceph-mon[36471]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T08:18:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:02 smithi049 ceph-mon[31942]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-16T08:18:02.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:02 smithi049 ceph-mon[31942]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-03-16T08:18:02.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:02 smithi129 ceph-mon[36561]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-16T08:18:02.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:02 smithi129 ceph-mon[36561]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-03-16T08:18:02.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:02 smithi154 ceph-mon[36471]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-16T08:18:02.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:02 smithi154 ceph-mon[36471]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-03-16T08:18:04.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:04 smithi049 ceph-mon[31942]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:18:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:04 smithi129 ceph-mon[36561]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:18:04.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:04 smithi154 ceph-mon[36471]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-16T08:18:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:06 smithi049 ceph-mon[31942]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:06 smithi129 ceph-mon[36561]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:06 smithi154 ceph-mon[36471]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 -- 172.21.15.49:0/2754573707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c105680 msgr2=0x7fde5c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 --2- 172.21.15.49:0/2754573707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c105680 0x7fde5c105a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde58009930 tx=0x7fde5802bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 -- 172.21.15.49:0/2754573707 shutdown_connections 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 --2- 172.21.15.49:0/2754573707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c1012f0 0x7fde5c10d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 --2- 172.21.15.49:0/2754573707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 --2- 172.21.15.49:0/2754573707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c105680 0x7fde5c105a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 -- 172.21.15.49:0/2754573707 >> 172.21.15.49:0/2754573707 conn(0x7fde5c077530 msgr2=0x7fde5c077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.614+0000 7fde63d76640 1 -- 172.21.15.49:0/2754573707 shutdown_connections 2024-03-16T08:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.615+0000 7fde63d76640 1 -- 172.21.15.49:0/2754573707 wait complete. 2024-03-16T08:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.615+0000 7fde63d76640 1 Processor -- start 2024-03-16T08:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.615+0000 7fde63d76640 1 -- start start 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c1012f0 0x7fde5c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c105680 0x7fde5c1a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c110ba0 con 0x7fde5c100930 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c110a20 con 0x7fde5c1012f0 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde63d76640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c110d20 con 0x7fde5c105680 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41428/0 (socket says 172.21.15.49:41428) 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 -- 172.21.15.49:0/2835619429 learned_addr learned my addr 172.21.15.49:0/2835619429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde62313640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c105680 0x7fde5c1a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61311640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c1012f0 0x7fde5c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 -- 172.21.15.49:0/2835619429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c105680 msgr2=0x7fde5c1a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c105680 0x7fde5c1a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 -- 172.21.15.49:0/2835619429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c1012f0 msgr2=0x7fde5c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c1012f0 0x7fde5c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 -- 172.21.15.49:0/2835619429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c196cf0 con 0x7fde5c100930 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.616+0000 7fde61b12640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde5802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.617+0000 7fde52ffd640 1 -- 172.21.15.49:0/2835619429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58041070 con 0x7fde5c100930 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.617+0000 7fde52ffd640 1 -- 172.21.15.49:0/2835619429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58004450 con 0x7fde5c100930 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.617+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5c196f80 con 0x7fde5c100930 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.617+0000 7fde52ffd640 1 -- 172.21.15.49:0/2835619429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58003e50 con 0x7fde5c100930 2024-03-16T08:18:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.617+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5c1974c0 con 0x7fde5c100930 2024-03-16T08:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.619+0000 7fde52ffd640 1 -- 172.21.15.49:0/2835619429 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fde58044070 con 0x7fde5c100930 2024-03-16T08:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.620+0000 7fde52ffd640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 0x7fde480a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.620+0000 7fde61311640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 0x7fde480a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.620+0000 7fde52ffd640 1 -- 172.21.15.49:0/2835619429 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fde580f0810 con 0x7fde5c100930 2024-03-16T08:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.620+0000 7fde61311640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 0x7fde480a5210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0699b0 tx=0x7fde4c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 Processor -- start 2024-03-16T08:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 -- start start 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 0x55a4b1565510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b1570c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4b14c9a40 con 0x55a4b156e820 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4b1472920 con 0x55a4b1565a50 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde66a33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4b14c5ae0 con 0x55a4b1565130 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde61311640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.621+0000 7fde62313640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b1570c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61b12640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 0x55a4b1565510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40894/0 (socket says 172.21.15.49:40894) 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 -- 172.21.15.49:0/59241827 learned_addr learned my addr 172.21.15.49:0/59241827 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 -- 172.21.15.49:0/59241827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 msgr2=0x55a4b1565510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 0x55a4b1565510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 -- 172.21.15.49:0/59241827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 msgr2=0x55a4b1570c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b1570c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 -- 172.21.15.49:0/59241827 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b148a460 con 0x55a4b1565a50 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61b12640 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 0x55a4b1565510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde61311640 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00e630 tx=0x7fde4c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00931b5f71bf6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde47fff640 1 -- 172.21.15.49:0/59241827 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c004020 con 0x55a4b1565a50 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde47fff640 1 -- 172.21.15.49:0/59241827 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c0041c0 con 0x55a4b1565a50 2024-03-16T08:18:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.622+0000 7fde47fff640 1 -- 172.21.15.49:0/59241827 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c01c7a0 con 0x55a4b1565a50 2024-03-16T08:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 -- 172.21.15.49:0/59241827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 msgr2=0x55a4b156e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00e630 tx=0x7fde4c014000 comp rx=0 tx=0).stop 2024-03-16T08:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 -- 172.21.15.49:0/59241827 shutdown_connections 2024-03-16T08:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b1570c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565a50 0x55a4b156e2e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 --2- 172.21.15.49:0/59241827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565130 0x55a4b1565510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 -- 172.21.15.49:0/59241827 >> 172.21.15.49:0/59241827 conn(0x55a4b155fa50 msgr2=0x55a4b155fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 -- 172.21.15.49:0/59241827 shutdown_connections 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.623+0000 7fde66a33740 1 -- 172.21.15.49:0/59241827 wait complete. 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.624+0000 7fde66a33740 1 -- start start 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.624+0000 7fde66a33740 1 Processor -- start 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.624+0000 7fde66a33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.624+0000 7fde66a33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b156bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.624+0000 7fde66a33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b156c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde66a33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4b1571450 con 0x55a4b156e820 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde66a33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a4b1485d70 con 0x55a4b1565130 2024-03-16T08:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde66a33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4b15715d0 con 0x55a4b1565a50 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde62313640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b156c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40902/0 (socket says 172.21.15.49:40902) 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 -- 172.21.15.49:0/4294389212 learned_addr learned my addr 172.21.15.49:0/4294389212 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61311640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b156bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde66a33740 10 client.0 _ll_register_callbacks cb 0x55a4b11dda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 -- 172.21.15.49:0/4294389212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 msgr2=0x55a4b156bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b156bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 -- 172.21.15.49:0/4294389212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 msgr2=0x55a4b156c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b156c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 -- 172.21.15.49:0/4294389212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b11de210 con 0x55a4b1565130 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c0041c0 con 0x55a4b1565130 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde61b12640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde580b9290 tx=0x7fde580b90d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a090b2ca7553be47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde457fa640 10 client.25679 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde457fa640 1 -- 172.21.15.49:0/4294389212 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5804b040 con 0x55a4b1565130 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.625+0000 7fde457fa640 1 -- 172.21.15.49:0/4294389212 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fde580c5290 con 0x55a4b1565130 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde457fa640 1 -- 172.21.15.49:0/4294389212 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fde580bce30 con 0x55a4b1565130 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 10 client.25679 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55a4b12aa620 con 0x55a4b1565130 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde457fa640 1 -- 172.21.15.49:0/4294389212 <== mon.2 v2:172.21.15.129:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fde580ee110 con 0x55a4b1565130 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 10 client.25679 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 20 client.25679 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b1580cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.626+0000 7fde66a33740 4 client.25679 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.627+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b14d6f80 con 0x55a4b157e8b0 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.627+0000 7fde61311640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b1580cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.627+0000 7fde457fa640 10 client.25679 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.627+0000 7fde61311640 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b1580cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23cf9e4d9806d7f server_cookie=367bc0c6136bc66b in_seq=0 out_seq=0 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde457fa640 1 -- 172.21.15.49:0/4294389212 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a4b14d6f80 con 0x55a4b157e8b0 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde457fa640 10 client.25679 handle_command_reply: tid=0 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 client.25679 shutdown 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 msgr2=0x55a4b156b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde580b9290 tx=0x7fde580b90d0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 shutdown_connections 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b1580cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b156e820 0x55a4b156c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde457fa640 0 client.25679 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b156bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 --2- 172.21.15.49:0/4294389212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b1565130 0x55a4b156b890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 >> 172.21.15.49:0/4294389212 conn(0x55a4b155fa50 msgr2=0x55a4b15617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 shutdown_connections 2024-03-16T08:18:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 1 -- 172.21.15.49:0/4294389212 wait complete. 2024-03-16T08:18:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.628+0000 7fde66a33740 20 client.25679 trim_cache size 0 max 16384 2024-03-16T08:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 Processor -- start 2024-03-16T08:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 -- start start 2024-03-16T08:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b156dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b157a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1b1e50 con 0x55a4b1565130 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1af720 con 0x55a4b156e820 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde66a33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1ad550 con 0x55a4b1565a50 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b156dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde61311640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b157a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b156dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41460/0 (socket says 172.21.15.49:41460) 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde61b12640 1 -- 172.21.15.49:0/304327570 learned_addr learned my addr 172.21.15.49:0/304327570 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.659+0000 7fde62313640 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 -- 172.21.15.49:0/304327570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 msgr2=0x55a4b15865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 -- 172.21.15.49:0/304327570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 msgr2=0x55a4b156dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b156dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 -- 172.21.15.49:0/304327570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b1564470 con 0x55a4b1565a50 2024-03-16T08:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde61311640 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b157a1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009730 tx=0x7fde4c00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944bb23c9a18dea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde477fe640 1 -- 172.21.15.49:0/304327570 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c03a070 con 0x55a4b1565a50 2024-03-16T08:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde477fe640 1 -- 172.21.15.49:0/304327570 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c003ad0 con 0x55a4b1565a50 2024-03-16T08:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.660+0000 7fde477fe640 1 -- 172.21.15.49:0/304327570 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0346f0 con 0x55a4b1565a50 2024-03-16T08:18:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- 172.21.15.49:0/304327570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 msgr2=0x55a4b157a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b157a1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009730 tx=0x7fde4c00eba0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- 172.21.15.49:0/304327570 shutdown_connections 2024-03-16T08:18:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b157a1f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 --2- 172.21.15.49:0/304327570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b156dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- 172.21.15.49:0/304327570 >> 172.21.15.49:0/304327570 conn(0x55a4b155fa50 msgr2=0x55a4b1561730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- 172.21.15.49:0/304327570 shutdown_connections 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- 172.21.15.49:0/304327570 wait complete. 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 -- start start 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.661+0000 7fde66a33740 1 Processor -- start 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b1568c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1adbe0 con 0x55a4b1565130 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1af050 con 0x55a4b156e820 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c1ae2b0 con 0x55a4b1565a50 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde66a33740 10 client.0 _ll_register_callbacks cb 0x7fde580bcc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41468/0 (socket says 172.21.15.49:41468) 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 -- 172.21.15.49:0/3779168228 learned_addr learned my addr 172.21.15.49:0/3779168228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde62313640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61311640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b1568c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 -- 172.21.15.49:0/3779168228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 msgr2=0x55a4b1568c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b1568c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 -- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 msgr2=0x55a4b15691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 -- 172.21.15.49:0/3779168228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b15e3ef0 con 0x55a4b1565130 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde62313640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde61b12640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fde580bcb30 tx=0x7fde58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95c3705e4d358c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.662+0000 7fde457fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5804b040 con 0x55a4b1565130 2024-03-16T08:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fde580b8d90 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c003ad0 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fde580c9030 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 10 client.15798 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4b15e3ef0 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c003ad0 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fde580c1df0 con 0x55a4b1565130 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 10 client.15798 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 20 client.15798 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.663+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b15e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.664+0000 7fde66a33740 4 client.15798 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.664+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4b14d12d0 con 0x55a4b157e8b0 2024-03-16T08:18:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.664+0000 7fde61311640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b15e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.664+0000 7fde457fa640 10 client.15798 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.664+0000 7fde61311640 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b15e7570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee642004256bd96 server_cookie=ebd81bbdc2adc154 in_seq=0 out_seq=0 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde457fa640 1 -- 172.21.15.49:0/3779168228 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4b14d12d0 con 0x55a4b157e8b0 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde457fa640 10 client.15798 handle_command_reply: tid=0 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 client.15798 shutdown 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 msgr2=0x55a4b1568740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fde580bcb30 tx=0x7fde58000bc0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 shutdown_connections 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a4b157e8b0 0x55a4b15e7570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4b156e820 0x55a4b15691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4b1565a50 0x55a4b1568c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 --2- 172.21.15.49:0/3779168228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4b1565130 0x55a4b1568740 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 >> 172.21.15.49:0/3779168228 conn(0x55a4b155fa50 msgr2=0x55a4b156b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde457fa640 0 client.15798 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 shutdown_connections 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.669+0000 7fde66a33740 1 -- 172.21.15.49:0/3779168228 wait complete. 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde66a33740 20 client.15798 trim_cache size 0 max 16384 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:18:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:18:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 msgr2=0x7fde480a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 0x7fde480a5210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0699b0 tx=0x7fde4c005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 msgr2=0x7fde5c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde5802cb30 comp rx=0 tx=0).stop 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 shutdown_connections 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fde480a2d50 0x7fde480a5210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde5c105680 0x7fde5c1a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5c1012f0 0x7fde5c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 --2- 172.21.15.49:0/2835619429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde5c100930 0x7fde5c198f00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 >> 172.21.15.49:0/2835619429 conn(0x7fde5c077530 msgr2=0x7fde5c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 shutdown_connections 2024-03-16T08:18:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:07.670+0000 7fde63d76640 1 -- 172.21.15.49:0/2835619429 wait complete. 2024-03-16T08:18:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:08 smithi049 ceph-mon[31942]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:18:08.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:08 smithi129 ceph-mon[36561]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:18:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:08 smithi154 ceph-mon[36471]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:18:09.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:18:09.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:18:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 -- 172.21.15.49:0/3256179911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 msgr2=0x7fad08107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 --2- 172.21.15.49:0/3256179911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad08107b60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7facf8009930 tx=0x7facf802bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 -- 172.21.15.49:0/3256179911 shutdown_connections 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 --2- 172.21.15.49:0/3256179911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad0810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 --2- 172.21.15.49:0/3256179911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 --2- 172.21.15.49:0/3256179911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad08107b60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 -- 172.21.15.49:0/3256179911 >> 172.21.15.49:0/3256179911 conn(0x7fad080fd280 msgr2=0x7fad080ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 -- 172.21.15.49:0/3256179911 shutdown_connections 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.339+0000 7fad0c84d640 1 -- 172.21.15.49:0/3256179911 wait complete. 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 Processor -- start 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 -- start start 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad081963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad0819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810f980 con 0x7fad08107780 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810f800 con 0x7fad08101770 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0c84d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810fb00 con 0x7fad08102130 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad0819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40932/0 (socket says 172.21.15.49:40932) 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad0659c640 1 -- 172.21.15.49:0/997486992 learned_addr learned my addr 172.21.15.49:0/997486992 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad05d9b640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad081963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 -- 172.21.15.49:0/997486992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 msgr2=0x7fad081963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad081963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 -- 172.21.15.49:0/997486992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 msgr2=0x7fad08195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.340+0000 7fad06d9d640 1 -- 172.21.15.49:0/997486992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0819ae50 con 0x7fad08107780 2024-03-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7fad0659c640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7fad05d9b640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad081963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7fad06d9d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad0819a720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad00009fd0 tx=0x7fad0000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7facf77fe640 1 -- 172.21.15.49:0/997486992 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0000ce20 con 0x7fad08107780 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7facf77fe640 1 -- 172.21.15.49:0/997486992 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00005020 con 0x7fad08107780 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0819b140 con 0x7fad08107780 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7facf77fe640 1 -- 172.21.15.49:0/997486992 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00010720 con 0x7fad08107780 2024-03-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.341+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a29d0 con 0x7fad08107780 2024-03-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.344+0000 7fad0f431740 1 Processor -- start 2024-03-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.344+0000 7facf77fe640 1 -- 172.21.15.49:0/997486992 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fad000108c0 con 0x7fad08107780 2024-03-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.344+0000 7fad0f431740 1 -- start start 2024-03-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 0x562e04049110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04049650 0x562e0404ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562e03fa4500 con 0x562e04040150 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562e03f4cbe0 con 0x562e04049650 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0f431740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e03fa05a0 con 0x562e04040a70 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad0659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 0x562e04049110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41486/0 (socket says 172.21.15.49:41486) 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 -- 172.21.15.49:0/3497842924 learned_addr learned my addr 172.21.15.49:0/3497842924 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04049650 0x562e0404ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7facf77fe640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 0x7facf00a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 -- 172.21.15.49:0/3497842924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 msgr2=0x562e04049110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 0x562e04049110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 -- 172.21.15.49:0/3497842924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04049650 msgr2=0x562e0404ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04049650 0x562e0404ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.345+0000 7fad05d9b640 1 -- 172.21.15.49:0/3497842924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e03f604b0 con 0x562e04040150 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7fad0659c640 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 0x562e04049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7facf77fe640 1 -- 172.21.15.49:0/997486992 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fad000cf630 con 0x7fad08107780 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7fad06d9d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 0x7facf00a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7fad05d9b640 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7facfc013800 tx=0x7facfc0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07260a348870011 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7facf4ff9640 1 -- 172.21.15.49:0/3497842924 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc013d80 con 0x562e04040150 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7facf4ff9640 1 -- 172.21.15.49:0/3497842924 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0130d0 con 0x562e04040150 2024-03-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.346+0000 7facf4ff9640 1 -- 172.21.15.49:0/3497842924 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc011910 con 0x562e04040150 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- 172.21.15.49:0/3497842924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 msgr2=0x562e04040530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad06d9d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 0x7facf00a9360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad000cccb0 tx=0x7fad00086040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7facfc013800 tx=0x7facfc0096e0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- 172.21.15.49:0/3497842924 shutdown_connections 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04049650 0x562e0404ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04040a70 0x562e04049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 --2- 172.21.15.49:0/3497842924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040150 0x562e04040530 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- 172.21.15.49:0/3497842924 >> 172.21.15.49:0/3497842924 conn(0x562e0403a880 msgr2=0x562e0403ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- 172.21.15.49:0/3497842924 shutdown_connections 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- 172.21.15.49:0/3497842924 wait complete. 2024-03-16T08:18:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 -- start start 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.347+0000 7fad0f431740 1 Processor -- start 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 0x562e04046650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e040470d0 0x562e040558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562e0404c3e0 con 0x562e04040a70 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562e03f60160 con 0x562e040470d0 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e0404c560 con 0x562e04049650 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0f431740 10 client.0 _ll_register_callbacks cb 0x562e03f44150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47386/0 (socket says 172.21.15.49:47386) 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 -- 172.21.15.49:0/2668371469 learned_addr learned my addr 172.21.15.49:0/2668371469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 0x562e04046650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 -- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e040470d0 msgr2=0x562e040558b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e040470d0 0x562e040558b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 -- 172.21.15.49:0/2668371469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 msgr2=0x562e04046650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 0x562e04046650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad0659c640 1 -- 172.21.15.49:0/2668371469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e03d827d0 con 0x562e04049650 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.348+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 0x562e04046650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7fad0659c640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facf8009430 tx=0x7facf802c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cc48ce8c5014c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8004040 con 0x562e04049650 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7facf80041e0 con 0x562e04049650 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc0130d0 con 0x562e04049650 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facf80043e0 con 0x562e04049650 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7fad0f431740 10 client.25690 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e03cb83f0 con 0x562e04049650 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0130d0 con 0x562e04049650 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7facf8003980 con 0x562e04049650 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.349+0000 7fad0f431740 10 client.25690 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad0f431740 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad0f431740 20 client.25690 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04059b70 0x562e0405bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad0f431740 4 client.25690 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e03fb1a40 con 0x562e04059b70 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.350+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04059b70 0x562e0405bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7facedffb640 10 client.25690 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04059b70 0x562e0405bff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd98a5226abacad server_cookie=7700e8ffdd82ee8 in_seq=0 out_seq=0 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7facedffb640 1 -- 172.21.15.49:0/2668371469 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562e03fb1a40 con 0x562e04059b70 2024-03-16T08:18:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7facedffb640 10 client.25690 handle_command_reply: tid=0 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 client.25690 shutdown 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 msgr2=0x562e04046b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facf8009430 tx=0x7facf802c880 comp rx=0 tx=0).stop 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 shutdown_connections 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04059b70 0x562e0405bff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e040470d0 0x562e040558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.351+0000 7facedffb640 0 client.25690 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e04049650 0x562e04046b90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 1 --2- 172.21.15.49:0/2668371469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e04040a70 0x562e04046650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 >> 172.21.15.49:0/2668371469 conn(0x562e0403a880 msgr2=0x562e0403c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 shutdown_connections 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 1 -- 172.21.15.49:0/2668371469 wait complete. 2024-03-16T08:18:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.352+0000 7fad0f431740 20 client.25690 trim_cache size 0 max 16384 2024-03-16T08:18:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 Processor -- start 2024-03-16T08:18:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 -- start start 2024-03-16T08:18:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e040618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e040470d0 0x562e04051800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7face8010f30 con 0x562e040470d0 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7face800e800 con 0x562e04040a70 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0f431740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7face800c630 con 0x562e0403a880 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47388/0 (socket says 172.21.15.49:47388) 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad0659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e040618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.369+0000 7fad05d9b640 1 -- 172.21.15.49:0/2701793372 learned_addr learned my addr 172.21.15.49:0/2701793372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e040470d0 0x562e04051800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 -- 172.21.15.49:0/2701793372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 msgr2=0x562e040618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e040618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 -- 172.21.15.49:0/2701793372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e040470d0 msgr2=0x562e04051800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e040470d0 0x562e04051800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 -- 172.21.15.49:0/2701793372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e03c780f0 con 0x562e0403a880 2024-03-16T08:18:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7fad05d9b640 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facfc00a010 tx=0x7facfc0115b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bfed41a7e70ca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7faceffff640 1 -- 172.21.15.49:0/2701793372 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc013150 con 0x562e0403a880 2024-03-16T08:18:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7faceffff640 1 -- 172.21.15.49:0/2701793372 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0051e0 con 0x562e0403a880 2024-03-16T08:18:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.370+0000 7faceffff640 1 -- 172.21.15.49:0/2701793372 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc02a810 con 0x562e0403a880 2024-03-16T08:18:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- 172.21.15.49:0/2701793372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 msgr2=0x562e04044fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facfc00a010 tx=0x7facfc0115b0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- 172.21.15.49:0/2701793372 shutdown_connections 2024-03-16T08:18:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e040470d0 0x562e04051800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e040618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 --2- 172.21.15.49:0/2701793372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e0403a880 0x562e04044fa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- 172.21.15.49:0/2701793372 >> 172.21.15.49:0/2701793372 conn(0x562e04046cf0 msgr2=0x562e04039070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- 172.21.15.49:0/2701793372 shutdown_connections 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- 172.21.15.49:0/2701793372 wait complete. 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 -- start start 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.371+0000 7fad0f431740 1 Processor -- start 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e0404e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e040470d0 0x562e0404ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7face800ccc0 con 0x562e0403a880 2024-03-16T08:18:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7face800e130 con 0x562e04040a70 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7face800d390 con 0x562e040470d0 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0f431740 10 client.0 _ll_register_callbacks cb 0x7facf8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41502/0 (socket says 172.21.15.49:41502) 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 -- 172.21.15.49:0/568610964 learned_addr learned my addr 172.21.15.49:0/568610964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0659c640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e0404e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e040470d0 0x562e0404ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 -- 172.21.15.49:0/568610964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e040470d0 msgr2=0x562e0404ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e040470d0 0x562e0404ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 -- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 msgr2=0x562e0404e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e0404e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 -- 172.21.15.49:0/568610964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e03f6b9d0 con 0x562e0403a880 2024-03-16T08:18:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad0659c640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e0404e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7fad05d9b640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7facfc013340 tx=0x7facfc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f5ae7a694f802c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7facedffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.372+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc02a790 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7facfc005390 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc0051e0 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facfc034020 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 10 client.15816 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e040c0a70 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0051e0 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7facfc00fa70 con 0x562e0403a880 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 10 client.15816 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 20 client.15816 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04049650 0x562e040c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 4 client.15816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e03fabd90 con 0x562e04049650 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.373+0000 7fad0659c640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04049650 0x562e040c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0659c640 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04049650 0x562e040c2fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7733dcb8008dd1 server_cookie=76afa5a5308b5454 in_seq=0 out_seq=0 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7facedffb640 10 client.15816 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7facedffb640 1 -- 172.21.15.49:0/568610964 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562e03fabd90 con 0x562e04049650 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7facedffb640 10 client.15816 handle_command_reply: tid=0 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 client.15816 shutdown 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 msgr2=0x562e04051810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7facfc013340 tx=0x7facfc02a3f0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 shutdown_connections 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562e04049650 0x562e040c2fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e040470d0 0x562e0404ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7facedffb640 0 client.15816 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562e04040a70 0x562e0404e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 --2- 172.21.15.49:0/568610964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e0403a880 0x562e04051810 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 >> 172.21.15.49:0/568610964 conn(0x562e04046cf0 msgr2=0x562e0403bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 shutdown_connections 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 1 -- 172.21.15.49:0/568610964 wait complete. 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.374+0000 7fad0f431740 20 client.15816 trim_cache size 0 max 16384 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:18:09.382 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 msgr2=0x7facf00a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 0x7facf00a9360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad000cccb0 tx=0x7fad00086040 comp rx=0 tx=0).stop 2024-03-16T08:18:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 msgr2=0x7fad0819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad0819a720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad00009fd0 tx=0x7fad0000ef10 comp rx=0 tx=0).stop 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 shutdown_connections 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a6ea0 0x7facf00a9360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08107780 0x7fad0819a720 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08102130 0x7fad081963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 --2- 172.21.15.49:0/997486992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08101770 0x7fad08195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 >> 172.21.15.49:0/997486992 conn(0x7fad080fd280 msgr2=0x7fad080fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 shutdown_connections 2024-03-16T08:18:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.375+0000 7fad0c84d640 1 -- 172.21.15.49:0/997486992 wait complete. 2024-03-16T08:18:09.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:18:09.390 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:18:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1001008544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41066f0 msgr2=0x7f2fb4106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1001008544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41066f0 0x7f2fb4106ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009860 tx=0x7f2fa402bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1001008544 shutdown_connections 2024-03-16T08:18:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1001008544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41010a0 0x7f2fb410d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1001008544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1001008544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41066f0 0x7f2fb4106ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1001008544 >> 172.21.15.49:0/1001008544 conn(0x7f2fb40777e0 msgr2=0x7f2fb4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1001008544 shutdown_connections 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.499+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1001008544 wait complete. 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.500+0000 7f2fb9aa1640 1 Processor -- start 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.500+0000 7f2fb9aa1640 1 -- start start 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.500+0000 7f2fb9aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.500+0000 7f2fb9aa1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41010a0 0x7f2fb41032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb9aa1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41066f0 0x7f2fb4103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb9aa1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4110b00 con 0x7f2fb41066f0 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb9aa1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4110980 con 0x7f2fb41010a0 2024-03-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb9aa1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4110c80 con 0x7f2fb41006e0 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47398/0 (socket says 172.21.15.49:47398) 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 -- 172.21.15.49:0/1928205069 learned_addr learned my addr 172.21.15.49:0/1928205069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb3fff640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41066f0 0x7f2fb4103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb2ffd640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41010a0 0x7f2fb41032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 -- 172.21.15.49:0/1928205069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41010a0 msgr2=0x7f2fb41032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41010a0 0x7f2fb41032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 -- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41066f0 msgr2=0x7f2fb4103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41066f0 0x7f2fb4103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 -- 172.21.15.49:0/1928205069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb41a65e0 con 0x7f2fb41006e0 2024-03-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb37fe640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4037040 tx=0x7f2fa40045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4041020 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.501+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa402c9f0 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.502+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb41a6870 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.502+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4039040 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.502+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a6d60 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.503+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb40685d0 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.503+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f2fa4044020 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.504+0000 7f2fb0ff9640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 0x7f2f9c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.504+0000 7f2fb2ffd640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 0x7f2f9c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.505+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2fa40f0780 con 0x7f2fb41006e0 2024-03-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.505+0000 7f2fb2ffd640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 0x7f2f9c0a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4104830 tx=0x7f2fa8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:09.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.508+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2fa404b020 con 0x7f2fb41006e0 2024-03-16T08:18:09.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.862+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fb4109c60 con 0x7f2fb41006e0 2024-03-16T08:18:09.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.863+0000 7f2fb0ff9640 1 -- 172.21.15.49:0/1928205069 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f2fa40bd3f0 con 0x7f2fb41006e0 2024-03-16T08:18:09.865 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:18:09.865 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:18:09.865 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 msgr2=0x7f2f9c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 0x7f2f9c0a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4104830 tx=0x7f2fa8005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 msgr2=0x7f2fb4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4037040 tx=0x7f2fa40045b0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 shutdown_connections 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2f9c0a2b10 0x7f2f9c0a4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fb41066f0 0x7f2fb4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fb41010a0 0x7f2fb41032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 --2- 172.21.15.49:0/1928205069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb41006e0 0x7f2fb4102da0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.866+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 >> 172.21.15.49:0/1928205069 conn(0x7f2fb40777e0 msgr2=0x7f2fb4068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.867+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 shutdown_connections 2024-03-16T08:18:09.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:09.867+0000 7f2fb9aa1640 1 -- 172.21.15.49:0/1928205069 wait complete. 2024-03-16T08:18:09.881 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:18:10.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.029+0000 7fb4db59e640 1 -- 172.21.15.49:0/2620370164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 msgr2=0x7fb4dc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.029+0000 7fb4db59e640 1 --2- 172.21.15.49:0/2620370164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc107a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009930 tx=0x7fb4cc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 -- 172.21.15.49:0/2620370164 shutdown_connections 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 --2- 172.21.15.49:0/2620370164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 --2- 172.21.15.49:0/2620370164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 --2- 172.21.15.49:0/2620370164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc107a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 -- 172.21.15.49:0/2620370164 >> 172.21.15.49:0/2620370164 conn(0x7fb4dc0fd180 msgr2=0x7fb4dc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 -- 172.21.15.49:0/2620370164 shutdown_connections 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 -- 172.21.15.49:0/2620370164 wait complete. 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 Processor -- start 2024-03-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.030+0000 7fb4db59e640 1 -- start start 2024-03-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc111a70 con 0x7fb4dc107660 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1118f0 con 0x7fb4dc101650 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4db59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc111bf0 con 0x7fb4dc102010 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41552/0 (socket says 172.21.15.49:41552) 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 -- 172.21.15.49:0/1086283400 learned_addr learned my addr 172.21.15.49:0/1086283400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4da59c640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 -- 172.21.15.49:0/1086283400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 msgr2=0x7fb4dc19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 -- 172.21.15.49:0/1086283400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 msgr2=0x7fb4dc19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4dad9d640 1 -- 172.21.15.49:0/1086283400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc19f150 con 0x7fb4dc107660 2024-03-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4da59c640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.031+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400eb10 tx=0x7fb4d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4cb7fe640 1 -- 172.21.15.49:0/1086283400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4004b20 con 0x7fb4dc107660 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc19f440 con 0x7fb4dc107660 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4cb7fe640 1 -- 172.21.15.49:0/1086283400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4004cc0 con 0x7fb4dc107660 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1a6cd0 con 0x7fb4dc107660 2024-03-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.032+0000 7fb4cb7fe640 1 -- 172.21.15.49:0/1086283400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4010760 con 0x7fb4dc107660 2024-03-16T08:18:10.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.035+0000 7fb4cb7fe640 1 -- 172.21.15.49:0/1086283400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fb4d4005020 con 0x7fb4dc107660 2024-03-16T08:18:10.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.035+0000 7fb4e39e0740 1 Processor -- start 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 -- start start 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4cb7fe640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 0x7fb4c40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 0x7fb4c40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f1affd490 con 0x556f1b09a340 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f1afa5c90 con 0x556f1b099a20 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556f1aff9530 con 0x556f1b0a2f20 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4cb7fe640 1 -- 172.21.15.49:0/1086283400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb4d40cf740 con 0x7fb4dc107660 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4da59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4da59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41006/0 (socket says 172.21.15.49:41006) 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4da59c640 1 -- 172.21.15.49:0/2029411386 learned_addr learned my addr 172.21.15.49:0/2029411386 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.036+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 -- 172.21.15.49:0/2029411386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 msgr2=0x556f1b0a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 -- 172.21.15.49:0/2029411386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 msgr2=0x556f1b0a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 -- 172.21.15.49:0/2029411386 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1b060500 con 0x556f1b099a20 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 0x7fb4c40a4f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00095f0 tx=0x7fb4d0013c60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4da59c640 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4159110877774df0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4c8ff9640 1 -- 172.21.15.49:0/2029411386 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc042070 con 0x556f1b099a20 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4c8ff9640 1 -- 172.21.15.49:0/2029411386 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc004a70 con 0x556f1b099a20 2024-03-16T08:18:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.037+0000 7fb4c8ff9640 1 -- 172.21.15.49:0/2029411386 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc005190 con 0x556f1b099a20 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 -- 172.21.15.49:0/2029411386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 msgr2=0x556f1b099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc009630 comp rx=0 tx=0).stop 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 -- 172.21.15.49:0/2029411386 shutdown_connections 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/2029411386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b099e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 -- 172.21.15.49:0/2029411386 >> 172.21.15.49:0/2029411386 conn(0x556f1b094150 msgr2=0x556f1b094530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.038+0000 7fb4e39e0740 1 -- 172.21.15.49:0/2029411386 shutdown_connections 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 -- 172.21.15.49:0/2029411386 wait complete. 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 -- start start 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 Processor -- start 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b099a20 0x556f1b09ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 0x556f1b0a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f1b0a5cb0 con 0x556f1b0a2f20 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f1afb9230 con 0x556f1b09a340 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556f1b0a5e30 con 0x556f1b099a20 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.039+0000 7fb4e39e0740 10 client.0 _ll_register_callbacks cb 0x556f1aff5010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 0x556f1b0a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41022/0 (socket says 172.21.15.49:41022) 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 0x556f1b0a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41562/0 (socket says 172.21.15.49:41562) 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/1302071819 learned_addr learned my addr 172.21.15.49:0/1302071819 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4da59c640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b099a20 0x556f1b09ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/1302071819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b099a20 msgr2=0x556f1b09ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b099a20 0x556f1b09ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/1302071819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 msgr2=0x556f1b0a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 0x556f1b0a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/1302071819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1afc8100 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000fdf0 tx=0x7fb4d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47ed8721570b3e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0026070 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb4d0020630 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.040+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc004a70 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb4d0032050 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 10 client.25718 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f1ad113f0 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc004a70 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb4d001f070 con 0x556f1b09a340 2024-03-16T08:18:10.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 10 client.25718 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 10 client.25718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 20 client.25718 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b0b32b0 0x556f1b0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 4 client.25718 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f1b00a9d0 con 0x556f1b0b32b0 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.041+0000 7fb4da59c640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b0b32b0 0x556f1b0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.042+0000 7fb4c1ffb640 10 client.25718 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.042+0000 7fb4da59c640 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b0b32b0 0x556f1b0b56d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2a2cf617332a1b4 server_cookie=10966796c27c8951 in_seq=0 out_seq=0 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/1302071819 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556f1b00a9d0 con 0x556f1b0b32b0 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4c1ffb640 10 client.25718 handle_command_reply: tid=0 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 client.25718 shutdown 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 msgr2=0x556f1b0a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000fdf0 tx=0x7fb4d0004730 comp rx=0 tx=0).stop 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 shutdown_connections 2024-03-16T08:18:10.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b0b32b0 0x556f1b0b56d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b0a2f20 0x556f1b0a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b09a340 0x556f1b0a0500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4c1ffb640 0 client.25718 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/1302071819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b099a20 0x556f1b09ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 >> 172.21.15.49:0/1302071819 conn(0x556f1b094150 msgr2=0x556f1b095ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 shutdown_connections 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 1 -- 172.21.15.49:0/1302071819 wait complete. 2024-03-16T08:18:10.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.043+0000 7fb4e39e0740 20 client.25718 trim_cache size 0 max 16384 2024-03-16T08:18:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 Processor -- start 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 -- start start 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1b1d50 con 0x556f1b09a340 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1af620 con 0x556f1b099a20 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ad450 con 0x556f1b0a2f20 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41576/0 (socket says 172.21.15.49:41576) 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/3732527328 learned_addr learned my addr 172.21.15.49:0/3732527328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4da59c640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.070+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 -- 172.21.15.49:0/3732527328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 msgr2=0x556f1b0bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 -- 172.21.15.49:0/3732527328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 msgr2=0x556f1b0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 -- 172.21.15.49:0/3732527328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1addb7d0 con 0x556f1b099a20 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4da59c640 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a26f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c170 tx=0x7fb4cc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caef4c291d34aea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4c3fff640 1 -- 172.21.15.49:0/3732527328 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc054070 con 0x556f1b099a20 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4c3fff640 1 -- 172.21.15.49:0/3732527328 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc004c70 con 0x556f1b099a20 2024-03-16T08:18:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.071+0000 7fb4c3fff640 1 -- 172.21.15.49:0/3732527328 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc042040 con 0x556f1b099a20 2024-03-16T08:18:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- 172.21.15.49:0/3732527328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 msgr2=0x556f1b0a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a26f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c170 tx=0x7fb4cc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- 172.21.15.49:0/3732527328 shutdown_connections 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/3732527328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a26f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- 172.21.15.49:0/3732527328 >> 172.21.15.49:0/3732527328 conn(0x556f1b094150 msgr2=0x556f1b095e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- 172.21.15.49:0/3732527328 shutdown_connections 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- 172.21.15.49:0/3732527328 wait complete. 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 -- start start 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.072+0000 7fb4e39e0740 1 Processor -- start 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1adae0 con 0x556f1b09a340 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41592/0 (socket says 172.21.15.49:41592) 2024-03-16T08:18:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/4216687061 learned_addr learned my addr 172.21.15.49:0/4216687061 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1aef50 con 0x556f1b099a20 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ae1b0 con 0x556f1b0a2f20 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4da59c640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4e39e0740 10 client.0 _ll_register_callbacks cb 0x7fb4d0012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4dad9d640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/4216687061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 msgr2=0x556f1b0a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 msgr2=0x556f1b0a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1b1185d0 con 0x556f1b09a340 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4da59c640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4d9d9b640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000e9d0 tx=0x7fb4d000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2b1b327f6206c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.073+0000 7fb4c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0026040 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb4d000ec10 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc004c70 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb4d003e050 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 10 client.15840 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f1b119890 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc004c70 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb4d001f070 con 0x556f1b09a340 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 10 client.15840 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 10 client.15840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 20 client.15840 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b119fe0 0x556f1b11c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 4 client.15840 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.074+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f1b004d20 con 0x556f1b119fe0 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4da59c640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b119fe0 0x556f1b11c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4c1ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4da59c640 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b119fe0 0x556f1b11c400 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deffa69048258c80 server_cookie=6ecb48b20cf99ef in_seq=0 out_seq=0 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4c1ffb640 1 -- 172.21.15.49:0/4216687061 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f1b004d20 con 0x556f1b119fe0 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4c1ffb640 10 client.15840 handle_command_reply: tid=0 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4e39e0740 1 client.15840 shutdown 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 msgr2=0x556f1b0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.075+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000e9d0 tx=0x7fb4d000f830 comp rx=0 tx=0).stop 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 shutdown_connections 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x556f1b119fe0 0x556f1b11c400 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f1b0a2f20 0x556f1b0a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f1b09a340 0x556f1b0a6f00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4c1ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 --2- 172.21.15.49:0/4216687061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f1b099a20 0x556f1b0a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 >> 172.21.15.49:0/4216687061 conn(0x556f1b094150 msgr2=0x556f1b09ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 shutdown_connections 2024-03-16T08:18:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 1 -- 172.21.15.49:0/4216687061 wait complete. 2024-03-16T08:18:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4e39e0740 20 client.15840 trim_cache size 0 max 16384 2024-03-16T08:18:10.083 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 msgr2=0x7fb4c40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 0x7fb4c40a4f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00095f0 tx=0x7fb4d0013c60 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 msgr2=0x7fb4dc19ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.076+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400eb10 tx=0x7fb4d400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 shutdown_connections 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb4c40a2a90 0x7fb4c40a4f50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4dc107660 0x7fb4dc19ea20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4dc102010 0x7fb4dc19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 --2- 172.21.15.49:0/1086283400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4dc101650 0x7fb4dc19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 >> 172.21.15.49:0/1086283400 conn(0x7fb4dc0fd180 msgr2=0x7fb4dc10cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 shutdown_connections 2024-03-16T08:18:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.077+0000 7fb4db59e640 1 -- 172.21.15.49:0/1086283400 wait complete. 2024-03-16T08:18:10.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3664015048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 msgr2=0x7ffb7810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3664015048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb7810d520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009ca0 tx=0x7ffb7002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3664015048 shutdown_connections 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3664015048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb7810d520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3664015048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3664015048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb78106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3664015048 >> 172.21.15.49:0/3664015048 conn(0x7ffb780777e0 msgr2=0x7ffb78077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3664015048 shutdown_connections 2024-03-16T08:18:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.201+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3664015048 wait complete. 2024-03-16T08:18:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.202+0000 7ffb7d2c6640 1 Processor -- start 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.202+0000 7ffb7d2c6640 1 -- start start 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb781a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78110a70 con 0x7ffb78100650 2024-03-16T08:18:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781108f0 con 0x7ffb78101010 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb7d2c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78110bf0 con 0x7ffb78106660 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41610/0 (socket says 172.21.15.49:41610) 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41054/0 (socket says 172.21.15.49:41054) 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb76ffd640 1 -- 172.21.15.49:0/3970845121 learned_addr learned my addr 172.21.15.49:0/3970845121 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb777fe640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb781a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 -- 172.21.15.49:0/3970845121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 msgr2=0x7ffb781a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb781a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 -- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 msgr2=0x7ffb78198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb767fc640 1 -- 172.21.15.49:0/3970845121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb781a4410 con 0x7ffb78101010 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.203+0000 7ffb777fe640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb781a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb767fc640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00e9f0 tx=0x7ffb6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb5ffff640 1 -- 172.21.15.49:0/3970845121 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00ccc0 con 0x7ffb78101010 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb5ffff640 1 -- 172.21.15.49:0/3970845121 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c010910 con 0x7ffb78101010 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb781a45e0 con 0x7ffb78101010 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb5ffff640 1 -- 172.21.15.49:0/3970845121 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c018960 con 0x7ffb78101010 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.204+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb781a4b30 con 0x7ffb78101010 2024-03-16T08:18:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.207+0000 7ffb5ffff640 1 -- 172.21.15.49:0/3970845121 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7ffb6c010ac0 con 0x7ffb78101010 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 Processor -- start 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 -- start start 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb5ffff640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 0x7ffb600a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a0759767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb777fe640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 0x7ffb600a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0758d1490 con 0x55a07596d830 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a075879c90 con 0x55a07596e150 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a0758cd530 con 0x55a075976d30 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb5ffff640 1 -- 172.21.15.49:0/3970845121 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffb6c0cf490 con 0x7ffb78101010 2024-03-16T08:18:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41616/0 (socket says 172.21.15.49:41616) 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb76ffd640 1 -- 172.21.15.49:0/1818110834 learned_addr learned my addr 172.21.15.49:0/1818110834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.208+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a0759767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 -- 172.21.15.49:0/1818110834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 msgr2=0x55a075979120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 -- 172.21.15.49:0/1818110834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 msgr2=0x55a07596dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 -- 172.21.15.49:0/1818110834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a075934500 con 0x55a07596e150 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb777fe640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 0x7ffb600a4f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb70005e30 tx=0x7ffb70005f50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a0759767f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ccab0 tx=0x7ffb6c09ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e091bd737e16132 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb5d7fa640 1 -- 172.21.15.49:0/1818110834 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0a4090 con 0x55a07596e150 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb5d7fa640 1 -- 172.21.15.49:0/1818110834 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c098010 con 0x55a07596e150 2024-03-16T08:18:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.209+0000 7ffb5d7fa640 1 -- 172.21.15.49:0/1818110834 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0862d0 con 0x55a07596e150 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1818110834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 msgr2=0x55a0759767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a0759767f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ccab0 tx=0x7ffb6c09ba60 comp rx=0 tx=0).stop 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1818110834 shutdown_connections 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a0759767f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1818110834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07596dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1818110834 >> 172.21.15.49:0/1818110834 conn(0x55a075967f60 msgr2=0x55a075968340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1818110834 shutdown_connections 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1818110834 wait complete. 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 -- start start 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.210+0000 7ffb7ff8f740 1 Processor -- start 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 0x55a0759742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075974810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a075979a60 con 0x55a07596e150 2024-03-16T08:18:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a07588d230 con 0x55a075976d30 2024-03-16T08:18:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a075979be0 con 0x55a07596d830 2024-03-16T08:18:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb7ff8f740 10 client.0 _ll_register_callbacks cb 0x55a0758c9010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb767fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 0x55a0759742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47502/0 (socket says 172.21.15.49:47502) 2024-03-16T08:18:10.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 -- 172.21.15.49:0/475910975 learned_addr learned my addr 172.21.15.49:0/475910975 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb777fe640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075974810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 -- 172.21.15.49:0/475910975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 msgr2=0x55a075974810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075974810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 -- 172.21.15.49:0/475910975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 msgr2=0x55a0759742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 0x55a0759742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb76ffd640 1 -- 172.21.15.49:0/475910975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a07589c100 con 0x55a07596d830 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.211+0000 7ffb767fc640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 0x55a0759742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb68012d40 tx=0x7ffb6800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec51b1431d564ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680120d0 con 0x55a07596d830 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffb68010430 con 0x55a07596d830 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6c098010 con 0x55a07596d830 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffb68012270 con 0x55a07596d830 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb7ff8f740 10 client.25729 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a0755e53f0 con 0x55a07596d830 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.212+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c098010 con 0x55a07596d830 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffb6801ca40 con 0x55a07596d830 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 10 client.25729 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 10 client.25729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 20 client.25729 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759842a0 0x55a075986710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 4 client.25729 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0758de9d0 con 0x55a0759842a0 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb767fc640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759842a0 0x55a075986710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb767fc640 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759842a0 0x55a075986710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de39125270e2f7c8 server_cookie=42844ae66c92a34f in_seq=0 out_seq=0 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.213+0000 7ffb5aff5640 10 client.25729 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb5aff5640 1 -- 172.21.15.49:0/475910975 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0758de9d0 con 0x55a0759842a0 2024-03-16T08:18:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb5aff5640 10 client.25729 handle_command_reply: tid=0 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 client.25729 shutdown 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 msgr2=0x55a075973d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb68012d40 tx=0x7ffb6800f790 comp rx=0 tx=0).stop 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 shutdown_connections 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759842a0 0x55a075986710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075974810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596e150 0x55a0759742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/475910975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596d830 0x55a075973d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 >> 172.21.15.49:0/475910975 conn(0x55a075967f60 msgr2=0x55a075969cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb5aff5640 0 client.25729 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 shutdown_connections 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.214+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/475910975 wait complete. 2024-03-16T08:18:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.215+0000 7ffb7ff8f740 20 client.25729 trim_cache size 0 max 16384 2024-03-16T08:18:10.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 Processor -- start 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 -- start start 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596e150 0x55a075982800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075997980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781adad0 con 0x55a07596d830 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781ab3a0 con 0x55a075976d30 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781a9210 con 0x55a07596e150 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41630/0 (socket says 172.21.15.49:41630) 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.237+0000 7ffb76ffd640 1 -- 172.21.15.49:0/2211914719 learned_addr learned my addr 172.21.15.49:0/2211914719 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb777fe640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075997980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb767fc640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596e150 0x55a075982800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 -- 172.21.15.49:0/2211914719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596e150 msgr2=0x55a075982800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596e150 0x55a075982800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 -- 172.21.15.49:0/2211914719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 msgr2=0x55a075997980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075997980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 -- 172.21.15.49:0/2211914719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a075986a80 con 0x55a07596d830 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb777fe640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075997980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb68009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3bc4296aa9b61f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb5cff9640 1 -- 172.21.15.49:0/2211914719 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68029030 con 0x55a07596d830 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb5cff9640 1 -- 172.21.15.49:0/2211914719 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb68010430 con 0x55a07596d830 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.238+0000 7ffb5cff9640 1 -- 172.21.15.49:0/2211914719 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6801c420 con 0x55a07596d830 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/2211914719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 msgr2=0x55a0759764c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb68009420 comp rx=0 tx=0).stop 2024-03-16T08:18:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/2211914719 shutdown_connections 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a075976d30 0x55a075997980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a07596e150 0x55a075982800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/2211914719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a0759764c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/2211914719 >> 172.21.15.49:0/2211914719 conn(0x55a075967f60 msgr2=0x55a07596f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/2211914719 shutdown_connections 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.239+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/2211914719 wait complete. 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 -- start start 2024-03-16T08:18:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 Processor -- start 2024-03-16T08:18:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a07597f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781a9860 con 0x55a07596d830 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781aacd0 con 0x55a07596e150 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb781a9f30 con 0x55a075976d30 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb7ff8f740 10 client.0 _ll_register_callbacks cb 0x7ffb68009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41638/0 (socket says 172.21.15.49:41638) 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb777fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 -- 172.21.15.49:0/1081416735 learned_addr learned my addr 172.21.15.49:0/1081416735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb767fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a07597f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 -- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 msgr2=0x55a075979f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 -- 172.21.15.49:0/1081416735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 msgr2=0x55a07597f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a07597f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.240+0000 7ffb76ffd640 1 -- 172.21.15.49:0/1081416735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a07596df80 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb76ffd640 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb68010600 tx=0x7ffb68010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278a5902b98aae14 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68029040 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb68010430 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffb6801c420 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffb6803c050 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb7ff8f740 10 client.15858 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a0756af7d0 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.241+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb68010430 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffb68037940 con 0x55a07596d830 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 10 client.15858 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 20 client.15858 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759f5d60 0x55a0759f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 4 client.15858 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0758d8d20 con 0x55a0759f5d60 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759f5d60 0x55a0759f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb767fc640 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759f5d60 0x55a0759f8150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8850e81c4e2bc376 server_cookie=3291ff9b54012c9f in_seq=0 out_seq=0 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb5aff5640 10 client.15858 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb5aff5640 1 -- 172.21.15.49:0/1081416735 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0758d8d20 con 0x55a0759f5d60 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb5aff5640 10 client.15858 handle_command_reply: tid=0 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.242+0000 7ffb7ff8f740 1 client.15858 shutdown 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 msgr2=0x55a07597eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb68010600 tx=0x7ffb68010da0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 shutdown_connections 2024-03-16T08:18:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a0759f5d60 0x55a0759f8150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a075976d30 0x55a075979f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb5aff5640 0 client.15858 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a07596e150 0x55a07597f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 --2- 172.21.15.49:0/1081416735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a07596d830 0x55a07597eb40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 >> 172.21.15.49:0/1081416735 conn(0x55a075967f60 msgr2=0x55a075969390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 shutdown_connections 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 1 -- 172.21.15.49:0/1081416735 wait complete. 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7ff8f740 20 client.15858 trim_cache size 0 max 16384 2024-03-16T08:18:10.250 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 msgr2=0x7ffb600a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 0x7ffb600a4f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb70005e30 tx=0x7ffb70005f50 comp rx=0 tx=0).stop 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 msgr2=0x7ffb78199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.243+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00e9f0 tx=0x7ffb6c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 shutdown_connections 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffb600a2a90 0x7ffb600a4f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb78106660 0x7ffb781a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb78101010 0x7ffb78199390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 --2- 172.21.15.49:0/3970845121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb78100650 0x7ffb78198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 >> 172.21.15.49:0/3970845121 conn(0x7ffb780777e0 msgr2=0x7ffb78102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 shutdown_connections 2024-03-16T08:18:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.244+0000 7ffb7d2c6640 1 -- 172.21.15.49:0/3970845121 wait complete. 2024-03-16T08:18:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:10 smithi049 ceph-mon[31942]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-16T08:18:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:10 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1928205069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:10.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:18:10.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 -- 172.21.15.49:0/1430515290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b00692e0 msgr2=0x7f72b010dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 --2- 172.21.15.49:0/1430515290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b00692e0 0x7f72b010dc40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009860 tx=0x7f72a402bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:10.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 -- 172.21.15.49:0/1430515290 shutdown_connections 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 --2- 172.21.15.49:0/1430515290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b010e180 0x7f72b0110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 --2- 172.21.15.49:0/1430515290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b00692e0 0x7f72b010dc40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 --2- 172.21.15.49:0/1430515290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00689c0 0x7f72b0068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 -- 172.21.15.49:0/1430515290 >> 172.21.15.49:0/1430515290 conn(0x7f72b00777e0 msgr2=0x7f72b0077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 -- 172.21.15.49:0/1430515290 shutdown_connections 2024-03-16T08:18:10.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.415+0000 7f72b4e59640 1 -- 172.21.15.49:0/1430515290 wait complete. 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 Processor -- start 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 -- start start 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 0x7f72b019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00692e0 0x7f72b019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 0x7f72b019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0113d40 con 0x7f72b00689c0 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 0x7f72b019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 0x7f72b019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41646/0 (socket says 172.21.15.49:41646) 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72ae59c640 1 -- 172.21.15.49:0/727042246 learned_addr learned my addr 172.21.15.49:0/727042246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72add9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00692e0 0x7f72b019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0113bc0 con 0x7f72b010e180 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0113ec0 con 0x7f72b00692e0 2024-03-16T08:18:10.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.416+0000 7f72aed9d640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 0x7f72b019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 -- 172.21.15.49:0/727042246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00692e0 msgr2=0x7f72b019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00692e0 0x7f72b019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 -- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 msgr2=0x7f72b019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 0x7f72b019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 -- 172.21.15.49:0/727042246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b019f2d0 con 0x7f72b010e180 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f72aed9d640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 0x7f72b019eba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005fd0 tx=0x7f72a800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f729f7fe640 1 -- 172.21.15.49:0/727042246 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004b20 con 0x7f72b010e180 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f729f7fe640 1 -- 172.21.15.49:0/727042246 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a8003e50 con 0x7f72b010e180 2024-03-16T08:18:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.417+0000 7f729f7fe640 1 -- 172.21.15.49:0/727042246 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800b570 con 0x7f72b010e180 2024-03-16T08:18:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.418+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b019f5c0 con 0x7f72b010e180 2024-03-16T08:18:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.418+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01a6e50 con 0x7f72b010e180 2024-03-16T08:18:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.420+0000 7f729f7fe640 1 -- 172.21.15.49:0/727042246 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f72a801d020 con 0x7f72b010e180 2024-03-16T08:18:10.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.421+0000 7f729f7fe640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 0x7f72980a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.421+0000 7f729f7fe640 1 -- 172.21.15.49:0/727042246 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f72a80ceed0 con 0x7f72b010e180 2024-03-16T08:18:10.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.421+0000 7f72ae59c640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 0x7f72980a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.421+0000 7f72b7a41740 1 Processor -- start 2024-03-16T08:18:10.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 -- start start 2024-03-16T08:18:10.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72ae59c640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 0x7f72980a4fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009c90 tx=0x7f72a0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679430c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x55667943c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556679394490 con 0x556679439d30 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55667933cc90 con 0x556679431150 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72b7a41740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556679390530 con 0x556679430830 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679430c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72add9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72add9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41122/0 (socket says 172.21.15.49:41122) 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72add9b640 1 -- 172.21.15.49:0/2354173905 learned_addr learned my addr 172.21.15.49:0/2354173905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.422+0000 7f72aed9d640 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x55667943c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 -- 172.21.15.49:0/2354173905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 msgr2=0x556679430c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679430c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 -- 172.21.15.49:0/2354173905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 msgr2=0x55667943c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x55667943c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 -- 172.21.15.49:0/2354173905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566793f7500 con 0x556679431150 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72ae59c640 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679430c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f72add9b640 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005e00 tx=0x7f72a4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7845f0a48e9a890 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f729cff9640 1 -- 172.21.15.49:0/2354173905 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4039040 con 0x556679431150 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f729cff9640 1 -- 172.21.15.49:0/2354173905 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4034cb0 con 0x556679431150 2024-03-16T08:18:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.423+0000 7f729cff9640 1 -- 172.21.15.49:0/2354173905 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a402c710 con 0x556679431150 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- 172.21.15.49:0/2354173905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 msgr2=0x5566794397f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005e00 tx=0x7f72a4009510 comp rx=0 tx=0).stop 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- 172.21.15.49:0/2354173905 shutdown_connections 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x55667943c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794397f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 --2- 172.21.15.49:0/2354173905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679430c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- 172.21.15.49:0/2354173905 >> 172.21.15.49:0/2354173905 conn(0x55667942af60 msgr2=0x55667942b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- 172.21.15.49:0/2354173905 shutdown_connections 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- 172.21.15.49:0/2354173905 wait complete. 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.424+0000 7f72b7a41740 1 -- start start 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 Processor -- start 2024-03-16T08:18:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x556679437200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x556679437740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55667943cac0 con 0x556679439d30 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556679350230 con 0x556679431150 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55667943cc40 con 0x556679430830 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47554/0 (socket says 172.21.15.49:47554) 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 -- 172.21.15.49:0/3345919295 learned_addr learned my addr 172.21.15.49:0/3345919295 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72b7a41740 10 client.0 _ll_register_callbacks cb 0x55667938c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72add9b640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x556679437200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72aed9d640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x556679437740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 -- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 msgr2=0x556679437200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x556679437200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 -- 172.21.15.49:0/3345919295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 msgr2=0x556679437740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.425+0000 7f72ae59c640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x556679437740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72ae59c640 1 -- 172.21.15.49:0/3345919295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55667935f100 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72aed9d640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x556679437740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72add9b640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x556679437200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72ae59c640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72a001ba10 tx=0x7f72a0013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d8c9e669fda5fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f7295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a000ed40 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f72a00273e0 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a4034cb0 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72a0009e20 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72b7a41740 10 client.25747 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566790a83f0 con 0x556679430830 2024-03-16T08:18:10.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.426+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4034cb0 con 0x556679430830 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72a0026070 con 0x556679430830 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 10 client.25747 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 20 client.25747 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x55667944c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 4 client.25747 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566793a19d0 con 0x556679449f90 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72add9b640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x55667944c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f72add9b640 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x55667944c3e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f01a8ff2ffa872 server_cookie=62f65707a9bfac6c in_seq=0 out_seq=0 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.427+0000 7f7295ffb640 10 client.25747 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f7295ffb640 1 -- 172.21.15.49:0/3345919295 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566793a19d0 con 0x556679449f90 2024-03-16T08:18:10.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f7295ffb640 10 client.25747 handle_command_reply: tid=0 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 client.25747 shutdown 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 msgr2=0x556679436cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72a001ba10 tx=0x7f72a0013040 comp rx=0 tx=0).stop 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 shutdown_connections 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x55667944c3e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679439d30 0x556679437740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f7295ffb640 0 client.25747 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x556679437200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 --2- 172.21.15.49:0/3345919295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x556679436cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 >> 172.21.15.49:0/3345919295 conn(0x55667942af60 msgr2=0x55667942ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 shutdown_connections 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.428+0000 7f72b7a41740 1 -- 172.21.15.49:0/3345919295 wait complete. 2024-03-16T08:18:10.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.429+0000 7f72b7a41740 20 client.25747 trim_cache size 0 max 16384 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.446+0000 7f72b7a41740 1 Processor -- start 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.446+0000 7f72b7a41740 1 -- start start 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x5566794393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 0x556679451cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01b1f30 con 0x556679431150 2024-03-16T08:18:10.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01af800 con 0x556679439d30 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72b7a41740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01ad630 con 0x556679430830 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72aed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 0x556679451cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41678/0 (socket says 172.21.15.49:41678) 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 -- 172.21.15.49:0/866341076 learned_addr learned my addr 172.21.15.49:0/866341076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72ae59c640 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x5566794393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 -- 172.21.15.49:0/866341076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 msgr2=0x5566794393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x5566794393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 -- 172.21.15.49:0/866341076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 msgr2=0x556679451cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 0x556679451cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72add9b640 1 -- 172.21.15.49:0/866341076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566791727d0 con 0x556679431150 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.447+0000 7f72aed9d640 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 0x556679451cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.448+0000 7f72add9b640 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f72a4032570 tx=0x7f72a402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18bce34bcf2322a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.448+0000 7f7297fff640 1 -- 172.21.15.49:0/866341076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4039040 con 0x556679431150 2024-03-16T08:18:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.448+0000 7f7297fff640 1 -- 172.21.15.49:0/866341076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4004030 con 0x556679431150 2024-03-16T08:18:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.448+0000 7f7297fff640 1 -- 172.21.15.49:0/866341076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4034e60 con 0x556679431150 2024-03-16T08:18:10.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- 172.21.15.49:0/866341076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 msgr2=0x556679445860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f72a4032570 tx=0x7f72a402e040 comp rx=0 tx=0).stop 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- 172.21.15.49:0/866341076 shutdown_connections 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679439d30 0x556679451cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679431150 0x556679445860 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 --2- 172.21.15.49:0/866341076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679430830 0x5566794393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- 172.21.15.49:0/866341076 >> 172.21.15.49:0/866341076 conn(0x55667942af60 msgr2=0x556679429750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- 172.21.15.49:0/866341076 shutdown_connections 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- 172.21.15.49:0/866341076 wait complete. 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 -- start start 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.449+0000 7f72b7a41740 1 Processor -- start 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 0x556679433630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41680/0 (socket says 172.21.15.49:41680) 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01adcc0 con 0x556679430830 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72ae59c640 1 -- 172.21.15.49:0/83542659 learned_addr learned my addr 172.21.15.49:0/83542659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01af130 con 0x556679431150 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01ae390 con 0x556679439d30 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72add9b640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 0x556679433630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72b7a41740 10 client.0 _ll_register_callbacks cb 0x7f72a001b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 -- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 msgr2=0x5566794330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 -- 172.21.15.49:0/83542659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 msgr2=0x556679441e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556679442660 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72ae59c640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72add9b640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f72aed9d640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 0x556679433630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72a80cc4b0 tx=0x7f72a809b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f8a81ec5e8d96f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.450+0000 7f7295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a801a040 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f72a80995f0 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a4004030 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72a8095e60 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f72b7a41740 10 client.25753 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566794afe20 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.451+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a80b4170 con 0x556679439d30 2024-03-16T08:18:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72a800b040 con 0x556679439d30 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 10 client.25753 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 20 client.25753 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x5566794b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 4 client.25753 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f72a4004030 con 0x556679449f90 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72ae59c640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x5566794b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f7295ffb640 10 client.25753 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f72ae59c640 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x5566794b25d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3886a73fbe82d5c8 server_cookie=e837428de70ba761 in_seq=0 out_seq=0 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f7295ffb640 1 -- 172.21.15.49:0/83542659 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f72a4004030 con 0x556679449f90 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.452+0000 7f7295ffb640 10 client.25753 handle_command_reply: tid=0 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 client.25753 shutdown 2024-03-16T08:18:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 msgr2=0x556679433630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 0x556679433630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72a80cc4b0 tx=0x7f72a809b3b0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 shutdown_connections 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556679449f90 0x5566794b25d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556679439d30 0x556679433630 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f7295ffb640 0 client.25753 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556679431150 0x5566794330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 --2- 172.21.15.49:0/83542659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556679430830 0x556679441e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 >> 172.21.15.49:0/83542659 conn(0x55667942af60 msgr2=0x556679437510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 shutdown_connections 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 1 -- 172.21.15.49:0/83542659 wait complete. 2024-03-16T08:18:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.453+0000 7f72b7a41740 20 client.25753 trim_cache size 0 max 16384 2024-03-16T08:18:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 msgr2=0x7f72980a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 0x7f72980a4fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009c90 tx=0x7f72a0005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 msgr2=0x7f72b019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 0x7f72b019eba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005fd0 tx=0x7f72a800c740 comp rx=0 tx=0).stop 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 shutdown_connections 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f72980a2b10 0x7f72980a4fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72b010e180 0x7f72b019eba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72b00692e0 0x7f72b019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 --2- 172.21.15.49:0/727042246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72b00689c0 0x7f72b019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 >> 172.21.15.49:0/727042246 conn(0x7f72b00777e0 msgr2=0x7f72b010f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 shutdown_connections 2024-03-16T08:18:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:10.454+0000 7f72b4e59640 1 -- 172.21.15.49:0/727042246 wait complete. 2024-03-16T08:18:10.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:10 smithi129 ceph-mon[36561]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-16T08:18:10.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:10 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1928205069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:10 smithi154 ceph-mon[36471]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-16T08:18:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:10 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1928205069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:11.469 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:18:11.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:18:11.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 -- 172.21.15.49:0/56302883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7820107830 msgr2=0x7f7820107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 --2- 172.21.15.49:0/56302883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7820107830 0x7f7820107c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7814009930 tx=0x7f781402bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 -- 172.21.15.49:0/56302883 shutdown_connections 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 --2- 172.21.15.49:0/56302883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200693a0 0x7f782010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 --2- 172.21.15.49:0/56302883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78200689e0 0x7f7820068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 --2- 172.21.15.49:0/56302883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7820107830 0x7f7820107c10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 -- 172.21.15.49:0/56302883 >> 172.21.15.49:0/56302883 conn(0x7f78200777e0 msgr2=0x7f7820077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 -- 172.21.15.49:0/56302883 shutdown_connections 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.583+0000 7f7827159640 1 -- 172.21.15.49:0/56302883 wait complete. 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f7827159640 1 Processor -- start 2024-03-16T08:18:11.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f7827159640 1 -- start start 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f7827159640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f7827159640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 0x7f78201987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f7827159640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f782019cb40 0x7f782019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.584+0000 7f781ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 0x7f78201987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41180/0 (socket says 172.21.15.49:41180) 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 -- 172.21.15.49:0/228874977 learned_addr learned my addr 172.21.15.49:0/228874977 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7820111b80 con 0x7f78200693a0 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7820111a00 con 0x7f78200689e0 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7820111d00 con 0x7f782019cb40 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f78256f6640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f782019cb40 0x7f782019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 -- 172.21.15.49:0/228874977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f782019cb40 msgr2=0x7f782019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f782019cb40 0x7f782019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 -- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 msgr2=0x7f78201987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 0x7f78201987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782019d630 con 0x7f78200689e0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f781ffff640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 0x7f78201987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7824ef5640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f781402efd0 tx=0x7f7814004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f781dffb640 1 -- 172.21.15.49:0/228874977 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814041070 con 0x7f78200689e0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f781dffb640 1 -- 172.21.15.49:0/228874977 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78140054e0 con 0x7f78200689e0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.585+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782019d8c0 con 0x7f78200689e0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.586+0000 7f781dffb640 1 -- 172.21.15.49:0/228874977 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814003bf0 con 0x7f78200689e0 2024-03-16T08:18:11.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.586+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201a5030 con 0x7f78200689e0 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.588+0000 7f7829e14740 1 Processor -- start 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.588+0000 7f7829e14740 1 -- start start 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 0x5555d45caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5555d452e0d0 con 0x5555d45d3da0 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5555d44d6c70 con 0x5555d45cb710 2024-03-16T08:18:11.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7829e14740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5555d452a170 con 0x5555d45caa30 2024-03-16T08:18:11.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f781dffb640 1 -- 172.21.15.49:0/228874977 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f7814044070 con 0x7f78200689e0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f781ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f781ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41706/0 (socket says 172.21.15.49:41706) 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f781ffff640 1 -- 172.21.15.49:0/421041883 learned_addr learned my addr 172.21.15.49:0/421041883 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f7824ef5640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 0x5555d45caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.589+0000 7f78256f6640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781dffb640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 0x7f78100a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 -- 172.21.15.49:0/421041883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 msgr2=0x5555d45caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 0x5555d45caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 -- 172.21.15.49:0/421041883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 msgr2=0x5555d45ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 -- 172.21.15.49:0/421041883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d430a7e0 con 0x5555d45d3da0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f7824ef5640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 0x5555d45caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f78256f6640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 0x7f78100a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781dffb640 1 -- 172.21.15.49:0/228874977 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f78140f0cd0 con 0x7f78200689e0 2024-03-16T08:18:11.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f781ffff640 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f780c0114f0 tx=0x7f780c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26effa800ec367e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.590+0000 7f780affd640 1 -- 172.21.15.49:0/421041883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c004020 con 0x5555d45d3da0 2024-03-16T08:18:11.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f780affd640 1 -- 172.21.15.49:0/421041883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c0041c0 con 0x5555d45d3da0 2024-03-16T08:18:11.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f780affd640 1 -- 172.21.15.49:0/421041883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00f930 con 0x5555d45d3da0 2024-03-16T08:18:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 -- 172.21.15.49:0/421041883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 msgr2=0x5555d45d6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f78256f6640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 0x7f78100a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7818005fd0 tx=0x7f7818004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f780c0114f0 tx=0x7f780c0119b0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 -- 172.21.15.49:0/421041883 shutdown_connections 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d3da0 0x5555d45d6190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45caa30 0x5555d45caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 --2- 172.21.15.49:0/421041883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 -- 172.21.15.49:0/421041883 >> 172.21.15.49:0/421041883 conn(0x5555d4591c50 msgr2=0x5555d45c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.591+0000 7f7829e14740 1 -- 172.21.15.49:0/421041883 shutdown_connections 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 -- 172.21.15.49:0/421041883 wait complete. 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 -- start start 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 Processor -- start 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.592+0000 7f7829e14740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45d18c0 0x5555d45e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7829e14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5555d45db020 con 0x5555d45caa30 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7829e14740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5555d44ea0d0 con 0x5555d45cb710 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7829e14740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5555d45db1a0 con 0x5555d45d18c0 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41714/0 (socket says 172.21.15.49:41714) 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 -- 172.21.15.49:0/2947545927 learned_addr learned my addr 172.21.15.49:0/2947545927 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f781ffff640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7829e14740 10 client.0 _ll_register_callbacks cb 0x5555d44b3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f78256f6640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45d18c0 0x5555d45e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 -- 172.21.15.49:0/2947545927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45d18c0 msgr2=0x5555d45e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45d18c0 0x5555d45e44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 -- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 msgr2=0x5555d45d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 -- 172.21.15.49:0/2947545927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d436af70 con 0x5555d45caa30 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f781ffff640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7824ef5640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78140bd240 tx=0x7f7814003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49536824e09f3c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f77fffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140c1150 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f78140b9920 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.593+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c0041c0 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78140bd2b0 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 10 client.15888 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5555d4202170 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c0041c0 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f78140eeb80 con 0x5555d45caa30 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 10 client.15888 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 20 client.15888 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d45e8620 0x5555d45eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 4 client.15888 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d453b610 con 0x5555d45e8620 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.594+0000 7f781ffff640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d45e8620 0x5555d45eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.595+0000 7f781ffff640 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d45e8620 0x5555d45eaab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8111c29ae5b9a018 server_cookie=8e79b387f60edb00 in_seq=0 out_seq=0 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.595+0000 7f77fffff640 10 client.15888 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.595+0000 7f77fffff640 1 -- 172.21.15.49:0/2947545927 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5555d453b610 con 0x5555d45e8620 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.595+0000 7f77fffff640 10 client.15888 handle_command_reply: tid=0 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.595+0000 7f7829e14740 1 client.15888 shutdown 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 msgr2=0x5555d45d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78140bd240 tx=0x7f7814003680 comp rx=0 tx=0).stop 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 shutdown_connections 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d45e8620 0x5555d45eaab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45d18c0 0x5555d45e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45cb710 0x5555d45d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f77fffff640 0 client.15888 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 --2- 172.21.15.49:0/2947545927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45caa30 0x5555d45d0e40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 >> 172.21.15.49:0/2947545927 conn(0x5555d4591c50 msgr2=0x5555d45c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 shutdown_connections 2024-03-16T08:18:11.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 1 -- 172.21.15.49:0/2947545927 wait complete. 2024-03-16T08:18:11.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.596+0000 7f7829e14740 20 client.15888 trim_cache size 0 max 16384 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 Processor -- start 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 -- start start 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78201ae040 con 0x5555d45d18c0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78201ab910 con 0x5555d45f08c0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7829e14740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78201a9740 con 0x5555d45cb710 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f781ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f781ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41722/0 (socket says 172.21.15.49:41722) 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f781ffff640 1 -- 172.21.15.49:0/3267209663 learned_addr learned my addr 172.21.15.49:0/3267209663 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.631+0000 7f78256f6640 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 -- 172.21.15.49:0/3267209663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 msgr2=0x5555d45f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 -- 172.21.15.49:0/3267209663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 msgr2=0x5555d45e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 -- 172.21.15.49:0/3267209663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d45c0d40 con 0x5555d45d18c0 2024-03-16T08:18:11.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f78256f6640 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f781ffff640 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f780c014040 tx=0x7f780c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326b895f1a62097a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f780a7fc640 1 -- 172.21.15.49:0/3267209663 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c004190 con 0x5555d45d18c0 2024-03-16T08:18:11.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f780a7fc640 1 -- 172.21.15.49:0/3267209663 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c005640 con 0x5555d45d18c0 2024-03-16T08:18:11.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.632+0000 7f780a7fc640 1 -- 172.21.15.49:0/3267209663 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00f5c0 con 0x5555d45d18c0 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- 172.21.15.49:0/3267209663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 msgr2=0x5555d45e3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f780c014040 tx=0x7f780c0096c0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- 172.21.15.49:0/3267209663 shutdown_connections 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45d18c0 0x5555d45e3e00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 --2- 172.21.15.49:0/3267209663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- 172.21.15.49:0/3267209663 >> 172.21.15.49:0/3267209663 conn(0x5555d4591c50 msgr2=0x5555d45cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- 172.21.15.49:0/3267209663 shutdown_connections 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- 172.21.15.49:0/3267209663 wait complete. 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 -- start start 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.633+0000 7f7829e14740 1 Processor -- start 2024-03-16T08:18:11.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45e0970 0x5555d45cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78201a9dd0 con 0x5555d45e0970 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78201ab240 con 0x5555d45f08c0 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78201aa4a0 con 0x5555d45cb710 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7829e14740 10 client.0 _ll_register_callbacks cb 0x7f781402cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f781ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7824ef5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47618/0 (socket says 172.21.15.49:47618) 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f78256f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45e0970 0x5555d45cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.634+0000 7f7824ef5640 1 -- 172.21.15.49:0/3018992329 learned_addr learned my addr 172.21.15.49:0/3018992329 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 -- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 msgr2=0x5555d45cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 -- 172.21.15.49:0/3018992329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45e0970 msgr2=0x5555d45cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45e0970 0x5555d45cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 -- 172.21.15.49:0/3018992329 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d45cdc30 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7824ef5640 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78140b9710 tx=0x7f7814005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4efd78068b0feb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f77fffff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140c1150 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f78140ee6e0 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c005640 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78140bd080 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7829e14740 10 client.25768 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5555d464f9c0 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.635+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c005640 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f78140bc700 con 0x5555d45cb710 2024-03-16T08:18:11.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 10 client.25768 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 20 client.25768 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d464fe20 0x5555d4652240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 4 client.25768 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555d4535960 con 0x5555d464fe20 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f781ffff640 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d464fe20 0x5555d4652240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f77fffff640 10 client.25768 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.636+0000 7f781ffff640 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d464fe20 0x5555d4652240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba494c3fdb3796c6 server_cookie=5e2b2449cc0bc295 in_seq=0 out_seq=0 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.637+0000 7f77fffff640 1 -- 172.21.15.49:0/3018992329 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555d4535960 con 0x5555d464fe20 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.637+0000 7f77fffff640 10 client.25768 handle_command_reply: tid=0 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.637+0000 7f7829e14740 1 client.25768 shutdown 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 msgr2=0x5555d45e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78140b9710 tx=0x7f7814005a20 comp rx=0 tx=0).stop 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 shutdown_connections 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5555d464fe20 0x5555d4652240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555d45e0970 0x5555d45cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f77fffff640 0 client.25768 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555d45f08c0 0x5555d45cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 --2- 172.21.15.49:0/3018992329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555d45cb710 0x5555d45e0430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 >> 172.21.15.49:0/3018992329 conn(0x5555d4591c50 msgr2=0x5555d45c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 shutdown_connections 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 1 -- 172.21.15.49:0/3018992329 wait complete. 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.638+0000 7f7829e14740 20 client.25768 trim_cache size 0 max 16384 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "9c70b127-54af-4694-9735-b2a9525642ee", 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:18:11.644 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 msgr2=0x7f78100a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 0x7f78100a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7818005fd0 tx=0x7f7818004ab0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 msgr2=0x7f7820198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f781402efd0 tx=0x7f7814004270 comp rx=0 tx=0).stop 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 shutdown_connections 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f78100a2a90 0x7f78100a4f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f782019cb40 0x7f782019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78200693a0 0x7f78201987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 --2- 172.21.15.49:0/228874977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78200689e0 0x7f7820198280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 >> 172.21.15.49:0/228874977 conn(0x7f78200777e0 msgr2=0x7f78200fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 shutdown_connections 2024-03-16T08:18:11.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.639+0000 7f7827159640 1 -- 172.21.15.49:0/228874977 wait complete. 2024-03-16T08:18:11.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:18:11.651 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:18:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/495618223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440692e0 msgr2=0x7f5f4410dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/495618223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440692e0 0x7f5f4410dc40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009900 tx=0x7f5f3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/495618223 shutdown_connections 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/495618223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f44110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/495618223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440692e0 0x7f5f4410dc40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/495618223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440689c0 0x7f5f44068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/495618223 >> 172.21.15.49:0/495618223 conn(0x7f5f440777e0 msgr2=0x7f5f44077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/495618223 shutdown_connections 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.761+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/495618223 wait complete. 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 Processor -- start 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 -- start start 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 0x7f5f4419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f4419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 0x7f5f4419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44113d40 con 0x7f5f440692e0 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44113bc0 con 0x7f5f440689c0 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41748/0 (socket says 172.21.15.49:41748) 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f43fff640 1 -- 172.21.15.49:0/324626633 learned_addr learned my addr 172.21.15.49:0/324626633 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.762+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44113ec0 con 0x7f5f4410e180 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f4921b640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f4419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 -- 172.21.15.49:0/324626633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 msgr2=0x7f5f4419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f4419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 -- 172.21.15.49:0/324626633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 msgr2=0x7f5f4419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 0x7f5f4419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 -- 172.21.15.49:0/324626633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4419f290 con 0x7f5f440692e0 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f4921b640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f4419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 0x7f5f4419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f43fff640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f3002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f41ffb640 1 -- 172.21.15.49:0/324626633 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3004d070 con 0x7f5f440692e0 2024-03-16T08:18:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f41ffb640 1 -- 172.21.15.49:0/324626633 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30040de0 con 0x7f5f440692e0 2024-03-16T08:18:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.763+0000 7f5f41ffb640 1 -- 172.21.15.49:0/324626633 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003850 con 0x7f5f440692e0 2024-03-16T08:18:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.764+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4419f520 con 0x7f5f440692e0 2024-03-16T08:18:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.764+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441027e0 con 0x7f5f440692e0 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 Processor -- start 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 -- start start 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eadaa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eadaca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608ead05500 con 0x5608eada1a70 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5608eacadbe0 con 0x5608eadaa650 2024-03-16T08:18:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4d864740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5608ead015a0 con 0x5608eada1150 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4921b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4921b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47648/0 (socket says 172.21.15.49:47648) 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f4921b640 1 -- 172.21.15.49:0/2951314917 learned_addr learned my addr 172.21.15.49:0/2951314917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.767+0000 7f5f41ffb640 1 -- 172.21.15.49:0/324626633 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f5f30050070 con 0x7f5f440692e0 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 -- 172.21.15.49:0/2951314917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 msgr2=0x5608eadaca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eadaca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 -- 172.21.15.49:0/2951314917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 msgr2=0x5608eadaa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eadaa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eadaa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 -- 172.21.15.49:0/2951314917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eacc14b0 con 0x5608eada1150 2024-03-16T08:18:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eadaa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f4921b640 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c011760 tx=0x7f5f3c011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f293f357d6c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f41ffb640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 0x7f5f340a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f2effd640 1 -- 172.21.15.49:0/2951314917 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004d20 con 0x5608eada1150 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f41ffb640 1 -- 172.21.15.49:0/324626633 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5f300f85a0 con 0x7f5f440692e0 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.768+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 0x7f5f340a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.769+0000 7f5f2effd640 1 -- 172.21.15.49:0/2951314917 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c00f9a0 con 0x5608eada1150 2024-03-16T08:18:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.769+0000 7f5f2effd640 1 -- 172.21.15.49:0/2951314917 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0044d0 con 0x5608eada1150 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- 172.21.15.49:0/2951314917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 msgr2=0x5608eada1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 0x7f5f340a4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5608eada0c20 tx=0x7f5f38005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c011760 tx=0x7f5f3c011c20 comp rx=0 tx=0).stop 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- 172.21.15.49:0/2951314917 shutdown_connections 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eadaca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eadaa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 --2- 172.21.15.49:0/2951314917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada1530 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- 172.21.15.49:0/2951314917 >> 172.21.15.49:0/2951314917 conn(0x5608ead9b880 msgr2=0x5608ead9bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- 172.21.15.49:0/2951314917 shutdown_connections 2024-03-16T08:18:11.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- 172.21.15.49:0/2951314917 wait complete. 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 -- start start 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.770+0000 7f5f4d864740 1 Processor -- start 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eada7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eada8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eada7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47656/0 (socket says 172.21.15.49:47656) 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f48a1a640 1 -- 172.21.15.49:0/483529180 learned_addr learned my addr 172.21.15.49:0/483529180 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f4921b640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eada8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.771+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608eadad440 con 0x5608eada1a70 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5608eacc1160 con 0x5608eadaa650 2024-03-16T08:18:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5608eadad5c0 con 0x5608eada1150 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f4d864740 10 client.0 _ll_register_callbacks cb 0x5608eaca5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 -- 172.21.15.49:0/483529180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 msgr2=0x5608eada76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 -- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 msgr2=0x5608eada8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eada8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eaae37d0 con 0x5608eada1a70 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f4921b640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eada8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f43fff640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eada7c30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5f300f5cd0 tx=0x7f5f300c0e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfad77af2c663909 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f23fff640 10 client.15903 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c00f9a0 con 0x5608eada1a70 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f23fff640 1 -- 172.21.15.49:0/483529180 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300c4520 con 0x5608eada1a70 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.772+0000 7f5f23fff640 1 -- 172.21.15.49:0/483529180 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5f300cce20 con 0x5608eada1a70 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f23fff640 1 -- 172.21.15.49:0/483529180 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5f30004c80 con 0x5608eada1a70 2024-03-16T08:18:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 10 client.15903 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=24}) -- 0x5608eaa193f0 con 0x5608eada1a70 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f23fff640 1 -- 172.21.15.49:0/483529180 <== mon.0 v2:172.21.15.49:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5f300ddbd0 con 0x5608eada1a70 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 10 client.15903 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 20 client.15903 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eadba320 0x5608eadbc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.773+0000 7f5f4d864740 4 client.15903 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.774+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608ead12a40 con 0x5608eadba320 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.774+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eadba320 0x5608eadbc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.774+0000 7f5f23fff640 10 client.15903 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.774+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eadba320 0x5608eadbc710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0baa820da00e322 server_cookie=4870507a69a743d0 in_seq=0 out_seq=0 2024-03-16T08:18:11.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f23fff640 1 -- 172.21.15.49:0/483529180 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5608ead12a40 con 0x5608eadba320 2024-03-16T08:18:11.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f23fff640 10 client.15903 handle_command_reply: tid=0 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 client.15903 shutdown 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 msgr2=0x5608eada7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eada7c30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5f300f5cd0 tx=0x7f5f300c0e60 comp rx=0 tx=0).stop 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 shutdown_connections 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eadba320 0x5608eadbc710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f23fff640 0 client.15903 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadaa650 0x5608eada8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1a70 0x5608eada7c30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 --2- 172.21.15.49:0/483529180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eada1150 0x5608eada76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 >> 172.21.15.49:0/483529180 conn(0x5608ead9b880 msgr2=0x5608ead9d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 shutdown_connections 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 1 -- 172.21.15.49:0/483529180 wait complete. 2024-03-16T08:18:11.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.775+0000 7f5f4d864740 20 client.15903 trim_cache size 0 max 16384 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 Processor -- start 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 -- start start 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1150 0x5608eada9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 0x5608ead9d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ada60 con 0x5608eada1150 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ab330 con 0x5608eadba320 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4d864740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441a9160 con 0x5608eadaa650 2024-03-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1150 0x5608eada9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47672/0 (socket says 172.21.15.49:47672) 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f4921b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 0x5608ead9d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 -- 172.21.15.49:0/3243817834 learned_addr learned my addr 172.21.15.49:0/3243817834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 -- 172.21.15.49:0/3243817834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 msgr2=0x5608ead9d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 0x5608ead9d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 -- 172.21.15.49:0/3243817834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1150 msgr2=0x5608eada9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1150 0x5608eada9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.793+0000 7f5f43fff640 1 -- 172.21.15.49:0/3243817834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ea9d90f0 con 0x5608eadaa650 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.794+0000 7f5f4921b640 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 0x5608ead9d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.794+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002c820 tx=0x7f5f300f7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a19529480ab4d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.794+0000 7f5f2e7fc640 1 -- 172.21.15.49:0/3243817834 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300c4990 con 0x5608eadaa650 2024-03-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.794+0000 7f5f2e7fc640 1 -- 172.21.15.49:0/3243817834 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c00f9a0 con 0x5608eadaa650 2024-03-16T08:18:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.794+0000 7f5f2e7fc640 1 -- 172.21.15.49:0/3243817834 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300cd2b0 con 0x5608eadaa650 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- 172.21.15.49:0/3243817834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 msgr2=0x5608eadb61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002c820 tx=0x7f5f300f7000 comp rx=0 tx=0).stop 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- 172.21.15.49:0/3243817834 shutdown_connections 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eadba320 0x5608ead9d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadb61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3243817834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eada1150 0x5608eada9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- 172.21.15.49:0/3243817834 >> 172.21.15.49:0/3243817834 conn(0x5608ead9b880 msgr2=0x5608ead9ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- 172.21.15.49:0/3243817834 shutdown_connections 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- 172.21.15.49:0/3243817834 wait complete. 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 -- start start 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.795+0000 7f5f4d864740 1 Processor -- start 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 0x5608eadaee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441a97f0 con 0x5608eadba320 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441aac60 con 0x5608eada1150 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441a9ec0 con 0x5608eadaa650 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4d864740 10 client.0 _ll_register_callbacks cb 0x7f5f3002ce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41286/0 (socket says 172.21.15.49:41286) 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 -- 172.21.15.49:0/3294886199 learned_addr learned my addr 172.21.15.49:0/3294886199 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4921b640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 0x5608eadaee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 -- 172.21.15.49:0/3294886199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 msgr2=0x5608eadae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 -- 172.21.15.49:0/3294886199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 msgr2=0x5608eadaee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 0x5608eadaee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f48a1a640 1 -- 172.21.15.49:0/3294886199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eaccc9d0 con 0x5608eada1150 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.796+0000 7f5f4921b640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 0x5608eadaee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f48a1a640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5608eadbceb0 tx=0x7f5f380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9394f454e218827f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f23fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3802ad90 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5f3802a380 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c00f9a0 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5f3802a580 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f4d864740 10 client.25793 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5608eae209c0 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.797+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c00f9a0 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5f3803c900 con 0x5608eada1150 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 10 client.25793 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 20 client.25793 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eae21040 0x5608eae23460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 4 client.25793 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608ead0cd90 con 0x5608eae21040 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eae21040 0x5608eae23460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f23fff640 10 client.25793 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f43fff640 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eae21040 0x5608eae23460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc15455800a585ad server_cookie=b2fe2c4ece2800a1 in_seq=0 out_seq=0 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f23fff640 1 -- 172.21.15.49:0/3294886199 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5608ead0cd90 con 0x5608eae21040 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.798+0000 7f5f23fff640 10 client.25793 handle_command_reply: tid=0 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 client.25793 shutdown 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 msgr2=0x5608eadae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5608eadbceb0 tx=0x7f5f380046c0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 shutdown_connections 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5608eae21040 0x5608eae23460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608eadba320 0x5608eadaee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608eadaa650 0x5608eadae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f23fff640 0 client.25793 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 --2- 172.21.15.49:0/3294886199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608eada1150 0x5608eadae3e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 >> 172.21.15.49:0/3294886199 conn(0x5608ead9b880 msgr2=0x5608eada7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 shutdown_connections 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 1 -- 172.21.15.49:0/3294886199 wait complete. 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.799+0000 7f5f4d864740 20 client.25793 trim_cache size 0 max 16384 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stdout: "9c70b127-54af-4694-9735-b2a9525642ee": { 2024-03-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "9c70b127-54af-4694-9735-b2a9525642ee", 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 msgr2=0x7f5f340a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 0x7f5f340a4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5608eada0c20 tx=0x7f5f38005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 msgr2=0x7f5f4419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f3002cb30 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 shutdown_connections 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5f340a2b10 0x7f5f340a4fd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f4410e180 0x7f5f4419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f440692e0 0x7f5f4419a7e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 --2- 172.21.15.49:0/324626633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f440689c0 0x7f5f4419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 >> 172.21.15.49:0/324626633 conn(0x7f5f440777e0 msgr2=0x7f5f4410f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 shutdown_connections 2024-03-16T08:18:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:11.800+0000 7f5f4ac7e640 1 -- 172.21.15.49:0/324626633 wait complete. 2024-03-16T08:18:11.816 INFO:tasks.cephfs.filesystem:scrub status for tag:9c70b127-54af-4694-9735-b2a9525642ee - {'path': '/', 'tag': '9c70b127-54af-4694-9735-b2a9525642ee', 'options': 'recursive,force'} 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:12 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:18:12.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:12.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:18:12.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:18:12.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:12 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:18:12.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:12.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:18:12.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:18:12.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:12.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:12 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:18:14.527 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:14 smithi154 ceph-mon[36471]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:14 smithi049 ceph-mon[31942]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:14 smithi129 ceph-mon[36561]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-16T08:18:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:16 smithi049 ceph-mon[31942]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:16 smithi129 ceph-mon[36561]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:16.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:16 smithi154 ceph-mon[36471]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-16T08:18:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:18 smithi049 ceph-mon[31942]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-16T08:18:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:18 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:18:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:18 smithi129 ceph-mon[36561]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-16T08:18:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:18 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:18:18.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:18 smithi154 ceph-mon[36471]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-16T08:18:18.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:18 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:18:19.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:19 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:19 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:19 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:18:19.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:19 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:18:19.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:19 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:19 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:19 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:19 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:18:19.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:19 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:18:19.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:19 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:19 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:19 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:19.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:19 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:18:19.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:19 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:18:19.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:19 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:18:20.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:20 smithi049 ceph-mon[31942]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.807+0000 7f348a7b9640 1 -- 172.21.15.49:0/2156526337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34841077a0 msgr2=0x7f3484107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.807+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2156526337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34841077a0 0x7f3484107b80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3474009890 tx=0x7f347402bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 -- 172.21.15.49:0/2156526337 shutdown_connections 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2156526337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3484102150 0x7f348410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2156526337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484101790 0x7f3484101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2156526337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34841077a0 0x7f3484107b80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 -- 172.21.15.49:0/2156526337 >> 172.21.15.49:0/2156526337 conn(0x7f34840fd260 msgr2=0x7f34840ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 -- 172.21.15.49:0/2156526337 shutdown_connections 2024-03-16T08:18:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 -- 172.21.15.49:0/2156526337 wait complete. 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 Processor -- start 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.808+0000 7f348a7b9640 1 -- start start 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484102150 0x7f3484198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3484111bb0 con 0x7f34841077a0 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3484111a30 con 0x7f3484102150 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f348a7b9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3484111d30 con 0x7f3484101790 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3488d56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3488d56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36572/0 (socket says 172.21.15.49:36572) 2024-03-16T08:18:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47872/0 (socket says 172.21.15.49:47872) 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3488d56640 1 -- 172.21.15.49:0/2485342690 learned_addr learned my addr 172.21.15.49:0/2485342690 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f34837fe640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484102150 0x7f3484198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.809+0000 7f3483fff640 1 -- 172.21.15.49:0/2485342690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484102150 msgr2=0x7f3484198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3483fff640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484102150 0x7f3484198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3483fff640 1 -- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 msgr2=0x7f348419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3483fff640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3483fff640 1 -- 172.21.15.49:0/2485342690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348419d020 con 0x7f3484101790 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3488d56640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f3483fff640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f347402c180 tx=0x7f34740045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f34817fa640 1 -- 172.21.15.49:0/2485342690 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474041020 con 0x7f3484101790 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f34817fa640 1 -- 172.21.15.49:0/2485342690 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f347402c9f0 con 0x7f3484101790 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f34817fa640 1 -- 172.21.15.49:0/2485342690 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474039040 con 0x7f3484101790 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348419d2b0 con 0x7f3484101790 2024-03-16T08:18:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.810+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a4ba0 con 0x7f3484101790 2024-03-16T08:18:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.813+0000 7f34817fa640 1 -- 172.21.15.49:0/2485342690 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f3474044020 con 0x7f3484101790 2024-03-16T08:18:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.814+0000 7f348d3a7740 1 Processor -- start 2024-03-16T08:18:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.814+0000 7f348d3a7740 1 -- start start 2024-03-16T08:18:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.814+0000 7f34817fa640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 0x7f346c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f3488d56640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 0x7f346c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f34817fa640 1 -- 172.21.15.49:0/2485342690 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f34740f0740 con 0x7f3484101790 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f3488d56640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 0x7f346c0a5070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f347c009a60 tx=0x7f347c0095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f3483fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f3483fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51186/0 (socket says 172.21.15.49:51186) 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f34837fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f34837fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4eb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47886/0 (socket says 172.21.15.49:47886) 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.815+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4ee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 -- 172.21.15.49:0/1487051735 learned_addr learned my addr 172.21.15.49:0/1487051735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c8d442a40 con 0x563c8d4ebc30 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3488d56640 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4ee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c8d3eb920 con 0x563c8d4ddaf0 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c8d43eae0 con 0x563c8d4de410 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 -- 172.21.15.49:0/1487051735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 msgr2=0x563c8d4eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 -- 172.21.15.49:0/1487051735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 msgr2=0x563c8d4ee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4ee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3483fff640 1 -- 172.21.15.49:0/1487051735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8d403460 con 0x563c8d4ddaf0 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.816+0000 7f3488d56640 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.817+0000 7f3483fff640 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34740bc970 tx=0x7f347402bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f256b744ecdd13a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.817+0000 7f346a7fc640 1 -- 172.21.15.49:0/1487051735 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740bcc70 con 0x563c8d4ddaf0 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.817+0000 7f346a7fc640 1 -- 172.21.15.49:0/1487051735 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34740c0dc0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.817+0000 7f346a7fc640 1 -- 172.21.15.49:0/1487051735 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740c4f80 con 0x563c8d4ddaf0 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 msgr2=0x563c8d4dded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34740bc970 tx=0x7f347402bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 shutdown_connections 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 --2- 172.21.15.49:0/1487051735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4dded0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 >> 172.21.15.49:0/1487051735 conn(0x563c8d4d8a50 msgr2=0x563c8d4d8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 shutdown_connections 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- 172.21.15.49:0/1487051735 wait complete. 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.818+0000 7f348d3a7740 1 -- start start 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 Processor -- start 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4de410 0x563c8d4e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c8d3eb920 con 0x563c8d4ebc30 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c8d3fed70 con 0x563c8d4de410 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c8d4ee620 con 0x563c8d4ddaf0 2024-03-16T08:18:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f3483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f3483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47898/0 (socket says 172.21.15.49:47898) 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f3483fff640 1 -- 172.21.15.49:0/3324991292 learned_addr learned my addr 172.21.15.49:0/3324991292 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.819+0000 7f348d3a7740 10 client.0 _ll_register_callbacks cb 0x563c8d156a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3488d56640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 -- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4de410 msgr2=0x563c8d4e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f34837fe640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4de410 0x563c8d4e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4de410 0x563c8d4e9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 -- 172.21.15.49:0/3324991292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 msgr2=0x563c8d4e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 -- 172.21.15.49:0/3324991292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8d157210 con 0x563c8d4ddaf0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3488d56640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f3483fff640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f34740c4ee0 tx=0x7f34740c5410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a3958d18d3ca4f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f34637fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740bcc70 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f34740db2c0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34740c0dc0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f34740db4c0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f348d3a7740 10 client.25798 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.820+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c8d223620 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34740bc0c0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f347403c030 con 0x563c8d4ddaf0 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 10 client.25798 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 20 client.25798 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d4fb730 0x563c8d4fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 4 client.25798 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34740c0dc0 con 0x563c8d4fb730 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.821+0000 7f34837fe640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d4fb730 0x563c8d4fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.822+0000 7f34637fe640 10 client.25798 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.822+0000 7f34837fe640 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d4fb730 0x563c8d4fdb20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbdd283999866eb9 server_cookie=d5cfd0ac25ddc186 in_seq=0 out_seq=0 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.822+0000 7f34637fe640 1 -- 172.21.15.49:0/3324991292 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f34740c0dc0 con 0x563c8d4fb730 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.822+0000 7f34637fe640 10 client.25798 handle_command_reply: tid=0 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.822+0000 7f348d3a7740 1 client.25798 shutdown 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 msgr2=0x563c8d4e8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f34740c4ee0 tx=0x7f34740c5410 comp rx=0 tx=0).stop 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 shutdown_connections 2024-03-16T08:18:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d4fb730 0x563c8d4fdb20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4de410 0x563c8d4e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3324991292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4ddaf0 0x563c8d4e8d30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 >> 172.21.15.49:0/3324991292 conn(0x563c8d4d8a50 msgr2=0x563c8d4da7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f34637fe640 0 client.25798 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 shutdown_connections 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 1 -- 172.21.15.49:0/3324991292 wait complete. 2024-03-16T08:18:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.823+0000 7f348d3a7740 20 client.25798 trim_cache size 0 max 16384 2024-03-16T08:18:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 Processor -- start 2024-03-16T08:18:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 -- start start 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f34841ad990 con 0x563c8d4ebc30 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34841ab260 con 0x563c8d4ddaf0 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f348d3a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f34841a9090 con 0x563c8d4de410 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f3483fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f3488d56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f3483fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51200/0 (socket says 172.21.15.49:51200) 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.853+0000 7f3483fff640 1 -- 172.21.15.49:0/2132075025 learned_addr learned my addr 172.21.15.49:0/2132075025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f34837fe640 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 -- 172.21.15.49:0/2132075025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 msgr2=0x563c8d4e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 -- 172.21.15.49:0/2132075025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 msgr2=0x563c8d4e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 -- 172.21.15.49:0/2132075025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8d4dd410 con 0x563c8d4ddaf0 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3488d56640 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3483fff640 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3474005e00 tx=0x7f34740095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b63e7a9ea70565 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.854+0000 7f3469ffb640 1 -- 172.21.15.49:0/2132075025 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740b8f10 con 0x563c8d4ddaf0 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.855+0000 7f3469ffb640 1 -- 172.21.15.49:0/2132075025 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34740b8d30 con 0x563c8d4ddaf0 2024-03-16T08:18:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.855+0000 7f3469ffb640 1 -- 172.21.15.49:0/2132075025 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740ee0e0 con 0x563c8d4ddaf0 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- 172.21.15.49:0/2132075025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 msgr2=0x563c8d4f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3474005e00 tx=0x7f34740095b0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- 172.21.15.49:0/2132075025 shutdown_connections 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 --2- 172.21.15.49:0/2132075025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f7240 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- 172.21.15.49:0/2132075025 >> 172.21.15.49:0/2132075025 conn(0x563c8d4d8a50 msgr2=0x563c8d4d7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- 172.21.15.49:0/2132075025 shutdown_connections 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- 172.21.15.49:0/2132075025 wait complete. 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.856+0000 7f348d3a7740 1 -- start start 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 Processor -- start 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f34841a9720 con 0x563c8d4ebc30 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34841aab90 con 0x563c8d4ddaf0 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.857+0000 7f348d3a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f34841a9df0 con 0x563c8d4de410 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f348d3a7740 10 client.0 _ll_register_callbacks cb 0x7f347c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47926/0 (socket says 172.21.15.49:47926) 2024-03-16T08:18:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 -- 172.21.15.49:0/3598058580 learned_addr learned my addr 172.21.15.49:0/3598058580 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f3483fff640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f3488d56640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 -- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 msgr2=0x563c8d4f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 -- 172.21.15.49:0/3598058580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 msgr2=0x563c8d4e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 -- 172.21.15.49:0/3598058580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8d4e6160 con 0x563c8d4de410 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f3488d56640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34837fe640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x563c8d4e8590 tx=0x7f3478004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d26f570bcdcd9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34637fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.858+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478021020 con 0x563c8d4de410 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f347801dd20 con 0x563c8d4de410 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34740b8d30 con 0x563c8d4de410 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f347802c050 con 0x563c8d4de410 2024-03-16T08:18:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 10 client.25804 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c8d5617c0 con 0x563c8d4de410 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34740b8d30 con 0x563c8d4de410 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3478012570 con 0x563c8d4de410 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 10 client.25804 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 20 client.25804 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d561fa0 0x563c8d5643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 4 client.25804 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.859+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c8d44ff80 con 0x563c8d561fa0 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.860+0000 7f3483fff640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d561fa0 0x563c8d5643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.860+0000 7f3483fff640 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d561fa0 0x563c8d5643c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7fac59281ab0c4 server_cookie=8980f3d2bc0d8aef in_seq=0 out_seq=0 2024-03-16T08:18:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.860+0000 7f34637fe640 10 client.25804 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:20.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:20 smithi129 ceph-mon[36561]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f34637fe640 1 -- 172.21.15.49:0/3598058580 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c8d44ff80 con 0x563c8d561fa0 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f34637fe640 10 client.25804 handle_command_reply: tid=0 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 client.25804 shutdown 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 msgr2=0x563c8d4e55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x563c8d4e8590 tx=0x7f3478004410 comp rx=0 tx=0).stop 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 shutdown_connections 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563c8d561fa0 0x563c8d5643c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c8d4ebc30 0x563c8d4e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f34637fe640 0 client.25804 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c8d4de410 0x563c8d4e55e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 --2- 172.21.15.49:0/3598058580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c8d4ddaf0 0x563c8d4f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 >> 172.21.15.49:0/3598058580 conn(0x563c8d4d8a50 msgr2=0x563c8d4d7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 shutdown_connections 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 1 -- 172.21.15.49:0/3598058580 wait complete. 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.863+0000 7f348d3a7740 20 client.25804 trim_cache size 0 max 16384 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:18:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 msgr2=0x7f346c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 0x7f346c0a5070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f347c009a60 tx=0x7f347c0095c0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 msgr2=0x7f3484198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f347402c180 tx=0x7f34740045b0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 shutdown_connections 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f346c0a2bb0 0x7f346c0a5070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34841077a0 0x7f348419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3484102150 0x7f3484198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 --2- 172.21.15.49:0/2485342690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3484101790 0x7f3484198030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 >> 172.21.15.49:0/2485342690 conn(0x7f34840fd260 msgr2=0x7f34840fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 shutdown_connections 2024-03-16T08:18:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:20.864+0000 7f348a7b9640 1 -- 172.21.15.49:0/2485342690 wait complete. 2024-03-16T08:18:20.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:20 smithi154 ceph-mon[36471]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T08:18:22.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:22 smithi049 ceph-mon[31942]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:22.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:22 smithi129 ceph-mon[36561]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:22.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:22 smithi154 ceph-mon[36471]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:24.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:24 smithi049 ceph-mon[31942]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:24.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:24 smithi129 ceph-mon[36561]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:24.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:24 smithi154 ceph-mon[36471]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-16T08:18:26.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:26 smithi049 ceph-mon[31942]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T08:18:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:26 smithi129 ceph-mon[36561]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T08:18:26.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:26 smithi154 ceph-mon[36471]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T08:18:28.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:28 smithi049 ceph-mon[31942]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:28 smithi129 ceph-mon[36561]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:28.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:28 smithi154 ceph-mon[36471]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:30.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:30 smithi049 ceph-mon[31942]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:30 smithi129 ceph-mon[36561]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:30 smithi154 ceph-mon[36471]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:32 smithi049 ceph-mon[31942]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:32.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:32 smithi129 ceph-mon[36561]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:32.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:32 smithi154 ceph-mon[36471]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:33.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:33 smithi129 ceph-mon[36561]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 -- 172.21.15.49:0/3464437265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 msgr2=0x7fb358107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 --2- 172.21.15.49:0/3464437265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb358107c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009930 tx=0x7fb34c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 -- 172.21.15.49:0/3464437265 shutdown_connections 2024-03-16T08:18:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 --2- 172.21.15.49:0/3464437265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358069800 0x7fb35810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 --2- 172.21.15.49:0/3464437265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358068e60 0x7fb3580692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 --2- 172.21.15.49:0/3464437265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb358107c90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 -- 172.21.15.49:0/3464437265 >> 172.21.15.49:0/3464437265 conn(0x7fb358077530 msgr2=0x7fb358077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 -- 172.21.15.49:0/3464437265 shutdown_connections 2024-03-16T08:18:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.958+0000 7fb35de14640 1 -- 172.21.15.49:0/3464437265 wait complete. 2024-03-16T08:18:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 Processor -- start 2024-03-16T08:18:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 -- start start 2024-03-16T08:18:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 0x7fb358197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358069800 0x7fb3581984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb358111ba0 con 0x7fb3581078b0 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb358111a20 con 0x7fb358069800 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb35de14640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb358111d20 con 0x7fb358068e60 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb357fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 0x7fb358197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb357fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39196/0 (socket says 172.21.15.49:39196) 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb357fff640 1 -- 172.21.15.49:0/3294559554 learned_addr learned my addr 172.21.15.49:0/3294559554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.959+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358069800 0x7fb3581984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 -- 172.21.15.49:0/3294559554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 msgr2=0x7fb358197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 0x7fb358197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 -- 172.21.15.49:0/3294559554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358069800 msgr2=0x7fb3581984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358069800 0x7fb3581984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 -- 172.21.15.49:0/3294559554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35819cf00 con 0x7fb3581078b0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 0x7fb358197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb357fff640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ea60 tx=0x7fb35000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb354ff9640 1 -- 172.21.15.49:0/3294559554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35000ce40 con 0x7fb3581078b0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb354ff9640 1 -- 172.21.15.49:0/3294559554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb350005020 con 0x7fb3581078b0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35819d1f0 con 0x7fb3581078b0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb354ff9640 1 -- 172.21.15.49:0/3294559554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350010690 con 0x7fb3581078b0 2024-03-16T08:18:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.960+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a4a80 con 0x7fb3581078b0 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.962+0000 7fb354ff9640 1 -- 172.21.15.49:0/3294559554 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fb350005520 con 0x7fb3581078b0 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 Processor -- start 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 -- start start 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 0x563dc9cec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9cec820 0x563dc9ceec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563dc9c47a40 con 0x563dc9ce3130 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563dc9bf0920 con 0x563dc9ce3a50 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3609fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563dc9c43ae0 con 0x563dc9cec820 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 0x563dc9cec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39198/0 (socket says 172.21.15.49:39198) 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb354ff9640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 0x7fb3400a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb356ffd640 1 -- 172.21.15.49:0/3466150718 learned_addr learned my addr 172.21.15.49:0/3466150718 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.963+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9cec820 0x563dc9ceec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb357fff640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 0x7fb3400a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 -- 172.21.15.49:0/3466150718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9cec820 msgr2=0x563dc9ceec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9cec820 0x563dc9ceec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb354ff9640 1 -- 172.21.15.49:0/3294559554 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb3500cf570 con 0x7fb3581078b0 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 -- 172.21.15.49:0/3466150718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 msgr2=0x563dc9ce3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 -- 172.21.15.49:0/3466150718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc9c08460 con 0x563dc9ce3a50 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 0x563dc9cec2e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009230 tx=0x7fb34c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a48e27844f52ae3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb357fff640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 0x7fb3400a4ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ccbf0 tx=0x7fb35000ef90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb33dffb640 1 -- 172.21.15.49:0/3466150718 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c042070 con 0x563dc9ce3a50 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb33dffb640 1 -- 172.21.15.49:0/3466150718 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34c004a70 con 0x563dc9ce3a50 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.964+0000 7fb33dffb640 1 -- 172.21.15.49:0/3466150718 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c004ef0 con 0x563dc9ce3a50 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- 172.21.15.49:0/3466150718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 msgr2=0x563dc9cec2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 0x563dc9cec2e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009230 tx=0x7fb34c005d50 comp rx=0 tx=0).stop 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- 172.21.15.49:0/3466150718 shutdown_connections 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9cec820 0x563dc9ceec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3a50 0x563dc9cec2e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3466150718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- 172.21.15.49:0/3466150718 >> 172.21.15.49:0/3466150718 conn(0x563dc9cdda50 msgr2=0x563dc9cdde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- 172.21.15.49:0/3466150718 shutdown_connections 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- 172.21.15.49:0/3466150718 wait complete. 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.965+0000 7fb3609fd740 1 -- start start 2024-03-16T08:18:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 Processor -- start 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9ce9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563dc9cef3f0 con 0x563dc9ce3130 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563dc9c03d70 con 0x563dc9cec820 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563dc9cef570 con 0x563dc9ce3a50 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3609fd740 10 client.0 _ll_register_callbacks cb 0x563dc995ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39210/0 (socket says 172.21.15.49:39210) 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 -- 172.21.15.49:0/13085015 learned_addr learned my addr 172.21.15.49:0/13085015 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb3577fe640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9ce9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 -- 172.21.15.49:0/13085015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 msgr2=0x563dc9ce9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9ce9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 -- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 msgr2=0x563dc9cea3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cea3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 -- 172.21.15.49:0/13085015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc995c210 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb356ffd640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb348009730 tx=0x7fb348015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8433667e04506d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.966+0000 7fb336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348018030 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb348004030 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34c004a70 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb34802a050 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 10 client.15936 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563dc9a28620 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34c004a70 con 0x563dc9ce3130 2024-03-16T08:18:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb348010c20 con 0x563dc9ce3130 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 10 client.15936 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 20 client.15936 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9cfca20 0x563dc9cfee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 4 client.15936 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dc9c54f80 con 0x563dc9cfca20 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.967+0000 7fb3577fe640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9cfca20 0x563dc9cfee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.968+0000 7fb336ffd640 10 client.15936 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.968+0000 7fb3577fe640 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9cfca20 0x563dc9cfee10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0ccf65a9d79119 server_cookie=e8027f4b57dfeccc in_seq=0 out_seq=0 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.968+0000 7fb336ffd640 1 -- 172.21.15.49:0/13085015 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563dc9c54f80 con 0x563dc9cfca20 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.968+0000 7fb336ffd640 10 client.15936 handle_command_reply: tid=0 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.968+0000 7fb3609fd740 1 client.15936 shutdown 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 msgr2=0x563dc9ce9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb348009730 tx=0x7fb348015040 comp rx=0 tx=0).stop 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 shutdown_connections 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9cfca20 0x563dc9cfee10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cea3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9ce9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb336ffd640 0 client.15936 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 --2- 172.21.15.49:0/13085015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3130 0x563dc9ce9920 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 >> 172.21.15.49:0/13085015 conn(0x563dc9cdda50 msgr2=0x563dc9cdf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 shutdown_connections 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 1 -- 172.21.15.49:0/13085015 wait complete. 2024-03-16T08:18:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.969+0000 7fb3609fd740 20 client.15936 trim_cache size 0 max 16384 2024-03-16T08:18:33.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:33 smithi154 ceph-mon[36471]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 Processor -- start 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 -- start start 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3130 0x563dc9cec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9cf8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581afb70 con 0x563dc9cec820 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581ad440 con 0x563dc9ce3130 2024-03-16T08:18:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3609fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581ab270 con 0x563dc9ce3a50 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9cf8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39226/0 (socket says 172.21.15.49:39226) 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 -- 172.21.15.49:0/2474186049 learned_addr learned my addr 172.21.15.49:0/2474186049 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3130 0x563dc9cec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 -- 172.21.15.49:0/2474186049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 msgr2=0x563dc9cf8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9cf8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 -- 172.21.15.49:0/2474186049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3130 msgr2=0x563dc9cec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3130 0x563dc9cec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.984+0000 7fb356ffd640 1 -- 172.21.15.49:0/2474186049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc9ce54f0 con 0x563dc9cec820 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.985+0000 7fb356ffd640 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x563dc9cfb870 tx=0x7fb34800ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb5c48b442155b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.985+0000 7fb33d7fa640 1 -- 172.21.15.49:0/2474186049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348018040 con 0x563dc9cec820 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.985+0000 7fb33d7fa640 1 -- 172.21.15.49:0/2474186049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb348009b00 con 0x563dc9cec820 2024-03-16T08:18:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.985+0000 7fb33d7fa640 1 -- 172.21.15.49:0/2474186049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348010a30 con 0x563dc9cec820 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.985+0000 7fb3609fd740 1 -- 172.21.15.49:0/2474186049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 msgr2=0x563dc9d04700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x563dc9cfb870 tx=0x7fb34800ff90 comp rx=0 tx=0).stop 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- 172.21.15.49:0/2474186049 shutdown_connections 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9cec820 0x563dc9d04700 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3a50 0x563dc9cf8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- 172.21.15.49:0/2474186049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9ce3130 0x563dc9cec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- 172.21.15.49:0/2474186049 >> 172.21.15.49:0/2474186049 conn(0x563dc9cdda50 msgr2=0x563dc9cdc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- 172.21.15.49:0/2474186049 shutdown_connections 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- 172.21.15.49:0/2474186049 wait complete. 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- start start 2024-03-16T08:18:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 Processor -- start 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3130 0x563dc9cf0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cf0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581ab900 con 0x563dc9ce3a50 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581acd70 con 0x563dc9cec820 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581abfd0 con 0x563dc9ce3130 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.986+0000 7fb3609fd740 10 client.0 _ll_register_callbacks cb 0x7fb348013d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39236/0 (socket says 172.21.15.49:39236) 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 -- 172.21.15.49:0/3033903746 learned_addr learned my addr 172.21.15.49:0/3033903746 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3130 0x563dc9cf0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cf0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 -- 172.21.15.49:0/3033903746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3130 msgr2=0x563dc9cf0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3130 0x563dc9cf0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 -- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 msgr2=0x563dc9cf0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cf0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 -- 172.21.15.49:0/3033903746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc9d61d30 con 0x563dc9ce3a50 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3577fe640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb34c02c170 tx=0x7fb34c02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed943c6107c2dcdf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c054020 con 0x563dc9ce3a50 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb34c050430 con 0x563dc9ce3a50 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb348009b00 con 0x563dc9ce3a50 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb34c05f050 con 0x563dc9ce3a50 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3609fd740 10 client.15942 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.987+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563dc9d62f80 con 0x563dc9ce3a50 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb348009b00 con 0x563dc9ce3a50 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb34c004ef0 con 0x563dc9ce3a50 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 10 client.15942 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 20 client.15942 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9d632e0 0x563dc9d656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 4 client.15942 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563dc9c4f2d0 con 0x563dc9d632e0 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9d632e0 0x563dc9d656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb336ffd640 10 client.15942 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.988+0000 7fb356ffd640 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9d632e0 0x563dc9d656d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfdcc2ae8d72f7a5 server_cookie=79824d954d5fb299 in_seq=0 out_seq=0 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb336ffd640 1 -- 172.21.15.49:0/3033903746 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563dc9c4f2d0 con 0x563dc9d632e0 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb336ffd640 10 client.15942 handle_command_reply: tid=0 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 client.15942 shutdown 2024-03-16T08:18:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 msgr2=0x563dc9cf0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb34c02c170 tx=0x7fb34c02c330 comp rx=0 tx=0).stop 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 shutdown_connections 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563dc9d632e0 0x563dc9d656d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563dc9cec820 0x563dc9cf0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dc9ce3a50 0x563dc9cf0560 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb336ffd640 0 client.15942 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 --2- 172.21.15.49:0/3033903746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563dc9ce3130 0x563dc9cf0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 >> 172.21.15.49:0/3033903746 conn(0x563dc9cdda50 msgr2=0x563dc9cdcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 shutdown_connections 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 1 -- 172.21.15.49:0/3033903746 wait complete. 2024-03-16T08:18:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.991+0000 7fb3609fd740 20 client.15942 trim_cache size 0 max 16384 2024-03-16T08:18:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:18:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:18:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:18:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 msgr2=0x7fb3400a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 0x7fb3400a4ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ccbf0 tx=0x7fb35000ef90 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 msgr2=0x7fb35819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ea60 tx=0x7fb35000ef20 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 shutdown_connections 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb3400a2b30 0x7fb3400a4ff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581078b0 0x7fb35819c7d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb358069800 0x7fb3581984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 --2- 172.21.15.49:0/3294559554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb358068e60 0x7fb358197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 >> 172.21.15.49:0/3294559554 conn(0x7fb358077530 msgr2=0x7fb3580fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 shutdown_connections 2024-03-16T08:18:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:33.992+0000 7fb35de14640 1 -- 172.21.15.49:0/3294559554 wait complete. 2024-03-16T08:18:34.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:33 smithi049 ceph-mon[31942]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T08:18:34.959 INFO:tasks.workunit.client.0.smithi049.stderr:Updating files: 93% (11528/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.749 INFO:tasks.workunit.client.0.smithi049.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: git switch -c 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr:Or undo this operation with: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: git switch - 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:35.750 INFO:tasks.workunit.client.0.smithi049.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T08:18:35.754 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-16T08:18:35.816 INFO:tasks.workunit.client.0.smithi049.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T08:18:35.819 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T08:18:35.819 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T08:18:36.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:35 smithi129 ceph-mon[36561]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:36.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:35 smithi154 ceph-mon[36471]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:35 smithi049 ceph-mon[31942]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:18:36.599 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T08:18:36.669 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T08:18:36.712 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T08:18:36.713 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T08:18:36.713 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T08:18:36.754 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T08:18:36.757 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:18:36.757 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-16T08:18:36.813 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-03-16T08:18:36.814 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-03-16T08:18:36.814 DEBUG:teuthology.orchestra.run.smithi049:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-16T08:18:36.875 INFO:tasks.workunit.client.0.smithi049.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-03-16T08:18:36.892 INFO:tasks.workunit.client.0.smithi049.stderr:--2024-03-16 08:18:36-- http://download.ceph.com/qa/fsync-tester.c 2024-03-16T08:18:36.893 INFO:tasks.workunit.client.0.smithi049.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-16T08:18:36.914 INFO:tasks.workunit.client.0.smithi049.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-16T08:18:36.936 INFO:tasks.workunit.client.0.smithi049.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-16T08:18:36.936 INFO:tasks.workunit.client.0.smithi049.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-03-16T08:18:36.937 INFO:tasks.workunit.client.0.smithi049.stderr:Saving to: ‘fsync-tester.c’ 2024-03-16T08:18:36.939 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:36.940 INFO:tasks.workunit.client.0.smithi049.stderr: 0K .. 100% 531M=0s 2024-03-16T08:18:36.940 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:36.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-03-16 08:18:36 (531 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-03-16T08:18:36.940 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-16T08:18:36.940 INFO:tasks.workunit.client.0.smithi049.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-03-16T08:18:37.011 INFO:tasks.workunit.client.0.smithi049.stderr:+ ./fsync-tester 2024-03-16T08:18:38.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:37 smithi129 ceph-mon[36561]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:18:38.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:37 smithi154 ceph-mon[36471]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:18:38.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:37 smithi049 ceph-mon[31942]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:18:40.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:39 smithi129 ceph-mon[36561]: pgmap v46: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 176 KiB/s wr, 9 op/s 2024-03-16T08:18:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:39 smithi154 ceph-mon[36471]: pgmap v46: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 176 KiB/s wr, 9 op/s 2024-03-16T08:18:40.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:39 smithi049 ceph-mon[31942]: pgmap v46: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 176 KiB/s wr, 9 op/s 2024-03-16T08:18:41.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:18:41.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:18:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 -- 172.21.15.49:0/507692984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41009c0 msgr2=0x7f1ef4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 --2- 172.21.15.49:0/507692984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41009c0 0x7f1ef4100e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009900 tx=0x7f1edc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 -- 172.21.15.49:0/507692984 shutdown_connections 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 --2- 172.21.15.49:0/507692984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4101380 0x7f1ef410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 --2- 172.21.15.49:0/507692984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41009c0 0x7f1ef4100e40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 --2- 172.21.15.49:0/507692984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef4106710 0x7f1ef4106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 -- 172.21.15.49:0/507692984 >> 172.21.15.49:0/507692984 conn(0x7f1ef40776c0 msgr2=0x7f1ef4077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 -- 172.21.15.49:0/507692984 shutdown_connections 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.933+0000 7f1ef93df640 1 -- 172.21.15.49:0/507692984 wait complete. 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 Processor -- start 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 -- start start 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 0x7f1ef4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 0x7f1ef4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 0x7f1ef41a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4110a20 con 0x7f1ef4101380 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41108a0 con 0x7f1ef41009c0 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef93df640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4110ba0 con 0x7f1ef4106710 2024-03-16T08:18:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 0x7f1ef4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 0x7f1ef41a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 0x7f1ef4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 0x7f1ef4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39264/0 (socket says 172.21.15.49:39264) 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.934+0000 7f1ef27fc640 1 -- 172.21.15.49:0/371521845 learned_addr learned my addr 172.21.15.49:0/371521845 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 -- 172.21.15.49:0/371521845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 msgr2=0x7f1ef4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 0x7f1ef4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 -- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 msgr2=0x7f1ef4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 0x7f1ef4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 -- 172.21.15.49:0/371521845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef41a43c0 con 0x7f1ef4106710 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 0x7f1ef4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 0x7f1ef41a3e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00ec50 tx=0x7f1eec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1eebfff640 1 -- 172.21.15.49:0/371521845 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec003b70 con 0x7f1ef4106710 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1eebfff640 1 -- 172.21.15.49:0/371521845 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eec003d10 con 0x7f1ef4106710 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef41a4590 con 0x7f1ef4106710 2024-03-16T08:18:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1eebfff640 1 -- 172.21.15.49:0/371521845 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec010890 con 0x7f1ef4106710 2024-03-16T08:18:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.935+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef41a4a60 con 0x7f1ef4106710 2024-03-16T08:18:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.937+0000 7f1eebfff640 1 -- 172.21.15.49:0/371521845 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f1eec003fc0 con 0x7f1ef4106710 2024-03-16T08:18:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1eebfff640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 0x7f1ee00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1efbfc7740 1 Processor -- start 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 0x7f1ee00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1eebfff640 1 -- 172.21.15.49:0/371521845 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1eec0cf690 con 0x7f1ef4106710 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1efbfc7740 1 -- start start 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 0x7f1ee00a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009f40 tx=0x7f1ee4005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.938+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca72a500 con 0x55e2ca7d3a60 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca6d2be0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca7265a0 con 0x55e2ca7c6730 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39266/0 (socket says 172.21.15.49:39266) 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 -- 172.21.15.49:0/3049241390 learned_addr learned my addr 172.21.15.49:0/3049241390 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 -- 172.21.15.49:0/3049241390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c6730 msgr2=0x55e2ca7d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 -- 172.21.15.49:0/3049241390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 msgr2=0x55e2ca7c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 -- 172.21.15.49:0/3049241390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ca6e64b0 con 0x55e2ca7d3a60 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0984c0 tx=0x7f1eec09b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13af488862c926cd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ee97fa640 1 -- 172.21.15.49:0/3049241390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0b3070 con 0x55e2ca7d3a60 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ee97fa640 1 -- 172.21.15.49:0/3049241390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eec009820 con 0x55e2ca7d3a60 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.939+0000 7f1ee97fa640 1 -- 172.21.15.49:0/3049241390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011690 con 0x55e2ca7d3a60 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 -- 172.21.15.49:0/3049241390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 msgr2=0x55e2ca7d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0984c0 tx=0x7f1eec09b8c0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 -- 172.21.15.49:0/3049241390 shutdown_connections 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d5e50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/3049241390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 -- 172.21.15.49:0/3049241390 >> 172.21.15.49:0/3049241390 conn(0x55e2ca7c0880 msgr2=0x55e2ca7c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 -- 172.21.15.49:0/3049241390 shutdown_connections 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.940+0000 7f1efbfc7740 1 -- 172.21.15.49:0/3049241390 wait complete. 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 -- start start 2024-03-16T08:18:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 Processor -- start 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca7d6850 con 0x55e2ca7c6730 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca6e6160 con 0x55e2ca7d3a60 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2ca7d69d0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1efbfc7740 10 client.0 _ll_register_callbacks cb 0x55e2ca6ca150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39268/0 (socket says 172.21.15.49:39268) 2024-03-16T08:18:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef27fc640 1 -- 172.21.15.49:0/875634400 learned_addr learned my addr 172.21.15.49:0/875634400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 msgr2=0x55e2ca7d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/875634400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 msgr2=0x55e2ca7d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/875634400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ca5087d0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.941+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d0ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4011ca0 tx=0x7f1ee4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9504a62df463f7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400fd90 con 0x55e2ca7c5e10 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1ee4028440 con 0x55e2ca7c5e10 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eec009820 con 0x55e2ca7c5e10 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1ee4028680 con 0x55e2ca7c5e10 2024-03-16T08:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 10 client.25834 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e2ca43e3f0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eec009820 con 0x55e2ca7c5e10 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1ee4026020 con 0x55e2ca7c5e10 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 10 client.25834 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 20 client.25834 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7e3ac0 0x55e2ca7e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 4 client.25834 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.942+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2ca737a40 con 0x55e2ca7e3ac0 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.943+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7e3ac0 0x55e2ca7e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.943+0000 7f1eda7fc640 10 client.25834 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.943+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7e3ac0 0x55e2ca7e5f20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bb26b08ee6cb91 server_cookie=5f0dfc67a235962a in_seq=0 out_seq=0 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1eda7fc640 1 -- 172.21.15.49:0/875634400 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e2ca737a40 con 0x55e2ca7e3ac0 2024-03-16T08:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1eda7fc640 10 client.25834 handle_command_reply: tid=0 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 client.25834 shutdown 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 msgr2=0x55e2ca7d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d0ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4011ca0 tx=0x7f1ee4004730 comp rx=0 tx=0).stop 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 shutdown_connections 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7e3ac0 0x55e2ca7e5f20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7d3a60 0x55e2ca7d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/875634400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d0ac0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1eda7fc640 0 client.25834 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 >> 172.21.15.49:0/875634400 conn(0x55e2ca7c0880 msgr2=0x55e2ca7c25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 shutdown_connections 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 1 -- 172.21.15.49:0/875634400 wait complete. 2024-03-16T08:18:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.944+0000 7f1efbfc7740 20 client.25834 trim_cache size 0 max 16384 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 Processor -- start 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 -- start start 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c6730 0x55e2ca7eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41adad0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41ab3a0 con 0x55e2ca7c6730 2024-03-16T08:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41a91d0 con 0x55e2ca7c0880 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1ef2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1ef2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41594/0 (socket says 172.21.15.49:41594) 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/2917528542 learned_addr learned my addr 172.21.15.49:0/2917528542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.962+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c6730 0x55e2ca7eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/2917528542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c6730 msgr2=0x55e2ca7eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c6730 0x55e2ca7eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/2917528542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c5e10 msgr2=0x55e2ca7df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 -- 172.21.15.49:0/2917528542 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ca3fe0f0 con 0x55e2ca7c0880 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee4011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f1fcf33a63ca46 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ee8ff9640 1 -- 172.21.15.49:0/2917528542 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4028440 con 0x55e2ca7c0880 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ee8ff9640 1 -- 172.21.15.49:0/2917528542 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4028d00 con 0x55e2ca7c0880 2024-03-16T08:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.963+0000 7f1ee8ff9640 1 -- 172.21.15.49:0/2917528542 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400fb90 con 0x55e2ca7c0880 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- 172.21.15.49:0/2917528542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 msgr2=0x55e2ca7d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee4011d20 comp rx=0 tx=0).stop 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- 172.21.15.49:0/2917528542 shutdown_connections 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c6730 0x55e2ca7eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/2917528542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d31f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- 172.21.15.49:0/2917528542 >> 172.21.15.49:0/2917528542 conn(0x55e2ca7cf700 msgr2=0x55e2ca7c2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- 172.21.15.49:0/2917528542 shutdown_connections 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- 172.21.15.49:0/2917528542 wait complete. 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 -- start start 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.964+0000 7f1efbfc7740 1 Processor -- start 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41a9860 con 0x55e2ca7c6730 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41aacd0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41a9f30 con 0x55e2ca7c0880 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1efbfc7740 10 client.0 _ll_register_callbacks cb 0x7f1ee4009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40132/0 (socket says 172.21.15.49:40132) 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39284/0 (socket says 172.21.15.49:39284) 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 -- 172.21.15.49:0/1142577928 learned_addr learned my addr 172.21.15.49:0/1142577928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 -- 172.21.15.49:0/1142577928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 msgr2=0x55e2ca7d78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 -- 172.21.15.49:0/1142577928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 msgr2=0x55e2ca7d83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 -- 172.21.15.49:0/1142577928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ca6f19d0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1edc005940 tx=0x7f1edc02cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d9ec2536d62acd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.965+0000 7f1eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc04ae60 con 0x55e2ca7c5e10 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee4028d00 con 0x55e2ca7c5e10 2024-03-16T08:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1edc04a580 con 0x55e2ca7c5e10 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1edc04a780 con 0x55e2ca7c5e10 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 10 client.25844 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e2ca84a250 con 0x55e2ca7c5e10 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4028d00 con 0x55e2ca7c5e10 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1edc004ec0 con 0x55e2ca7c5e10 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 10 client.25844 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 20 client.25844 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7d3a60 0x55e2ca84c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 4 client.25844 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2ca731d90 con 0x55e2ca7d3a60 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.966+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7d3a60 0x55e2ca84c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7d3a60 0x55e2ca84c910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04ff1465f969a84 server_cookie=700b11818d2a0f1b in_seq=0 out_seq=0 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1eda7fc640 10 client.25844 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1eda7fc640 1 -- 172.21.15.49:0/1142577928 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e2ca731d90 con 0x55e2ca7d3a60 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1eda7fc640 10 client.25844 handle_command_reply: tid=0 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 client.25844 shutdown 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 msgr2=0x55e2ca7d7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1edc005940 tx=0x7f1edc02cda0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 shutdown_connections 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e2ca7d3a60 0x55e2ca84c910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ca7c6730 0x55e2ca7d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1eda7fc640 0 client.25844 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2ca7c5e10 0x55e2ca7d7e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 --2- 172.21.15.49:0/1142577928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2ca7c0880 0x55e2ca7d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 >> 172.21.15.49:0/1142577928 conn(0x55e2ca7cf700 msgr2=0x55e2ca7bf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 shutdown_connections 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 1 -- 172.21.15.49:0/1142577928 wait complete. 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.967+0000 7f1efbfc7740 20 client.25844 trim_cache size 0 max 16384 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:18:41.975 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 msgr2=0x7f1ee00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 0x7f1ee00a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009f40 tx=0x7f1ee4005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 msgr2=0x7f1ef41a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 0x7f1ef41a3e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00ec50 tx=0x7f1eec00c6a0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 shutdown_connections 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1ee00a2a90 0x7f1ee00a4f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ef4106710 0x7f1ef41a3e80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4101380 0x7f1ef4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 --2- 172.21.15.49:0/371521845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef41009c0 0x7f1ef4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 >> 172.21.15.49:0/371521845 conn(0x7f1ef40776c0 msgr2=0x7f1ef40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 shutdown_connections 2024-03-16T08:18:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:41.968+0000 7f1ef93df640 1 -- 172.21.15.49:0/371521845 wait complete. 2024-03-16T08:18:41.986 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:18:41.986 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- 172.21.15.49:0/4162249863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 msgr2=0x7fb284101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/4162249863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb284101c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb280009860 tx=0x7fb28002bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- 172.21.15.49:0/4162249863 shutdown_connections 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/4162249863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb28410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/4162249863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb284101c00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/4162249863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb284107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- 172.21.15.49:0/4162249863 >> 172.21.15.49:0/4162249863 conn(0x7fb2840fd270 msgr2=0x7fb2840ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- 172.21.15.49:0/4162249863 shutdown_connections 2024-03-16T08:18:42.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- 172.21.15.49:0/4162249863 wait complete. 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 Processor -- start 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.099+0000 7fb28b8c0640 1 -- start start 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb28419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb284111b60 con 0x7fb284107790 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841119e0 con 0x7fb284101780 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28b8c0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb284111ce0 con 0x7fb284102140 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb289e5d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb28419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28965c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28965c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40140/0 (socket says 172.21.15.49:40140) 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41638/0 (socket says 172.21.15.49:41638) 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb289e5d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb28419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39304/0 (socket says 172.21.15.49:39304) 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28965c640 1 -- 172.21.15.49:0/1119981324 learned_addr learned my addr 172.21.15.49:0/1119981324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 -- 172.21.15.49:0/1119981324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 msgr2=0x7fb2841980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 -- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 msgr2=0x7fb28419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb28419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 -- 172.21.15.49:0/1119981324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28419d040 con 0x7fb284102140 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb28965c640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.100+0000 7fb288e5b640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb28002c0b0 tx=0x7fb2800045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.101+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280041020 con 0x7fb284102140 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.101+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28002c9f0 con 0x7fb284102140 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.101+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28419d2d0 con 0x7fb284102140 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.101+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280039040 con 0x7fb284102140 2024-03-16T08:18:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.101+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2841a4bc0 con 0x7fb284102140 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.102+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2840685d0 con 0x7fb284102140 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.103+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fb280044020 con 0x7fb284102140 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.103+0000 7fb27a7fc640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 0x7fb2700a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.104+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb2800f0920 con 0x7fb284102140 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.104+0000 7fb28965c640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 0x7fb2700a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.104+0000 7fb28965c640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 0x7fb2700a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2740096c0 tx=0x7fb274009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.107+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb28004b020 con 0x7fb284102140 2024-03-16T08:18:42.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:41 smithi129 ceph-mon[36561]: pgmap v47: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:42.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:41 smithi154 ceph-mon[36471]: pgmap v47: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:42.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:41 smithi049 ceph-mon[31942]: pgmap v47: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.333+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2841993a0 con 0x7fb284102140 2024-03-16T08:18:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.334+0000 7fb27a7fc640 1 -- 172.21.15.49:0/1119981324 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fb2800bd550 con 0x7fb284102140 2024-03-16T08:18:42.335 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:18:42.335 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:18:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:18:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 msgr2=0x7fb2700a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 0x7fb2700a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2740096c0 tx=0x7fb274009260 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 msgr2=0x7fb2841985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb28002c0b0 tx=0x7fb2800045b0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 shutdown_connections 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb2700a2a90 0x7fb2700a4f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb284107790 0x7fb28419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb284102140 0x7fb2841985e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 --2- 172.21.15.49:0/1119981324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb284101780 0x7fb2841980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 >> 172.21.15.49:0/1119981324 conn(0x7fb2840fd270 msgr2=0x7fb2840fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 shutdown_connections 2024-03-16T08:18:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.336+0000 7fb28b8c0640 1 -- 172.21.15.49:0/1119981324 wait complete. 2024-03-16T08:18:42.349 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:18:42.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/4200899135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 msgr2=0x7f2b68110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/4200899135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b68110570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009be0 tx=0x7f2b6002bef0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/4200899135 shutdown_connections 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/4200899135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b68110570 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/4200899135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680692e0 0x7f2b6810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/4200899135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680689c0 0x7f2b68068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/4200899135 >> 172.21.15.49:0/4200899135 conn(0x7f2b680777e0 msgr2=0x7f2b68077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/4200899135 shutdown_connections 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.464+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/4200899135 wait complete. 2024-03-16T08:18:42.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 Processor -- start 2024-03-16T08:18:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 -- start start 2024-03-16T08:18:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 0x7f2b6819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68113d40 con 0x7f2b680689c0 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68113bc0 con 0x7f2b680692e0 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b6dc9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68113ec0 con 0x7f2b6810e180 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 0x7f2b6819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 0x7f2b6819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39326/0 (socket says 172.21.15.49:39326) 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40148/0 (socket says 172.21.15.49:40148) 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b677fe640 1 -- 172.21.15.49:0/212284871 learned_addr learned my addr 172.21.15.49:0/212284871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.465+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41650/0 (socket says 172.21.15.49:41650) 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 -- 172.21.15.49:0/212284871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 msgr2=0x7f2b6819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 -- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 msgr2=0x7f2b6819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 0x7f2b6819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 -- 172.21.15.49:0/212284871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6819f1c0 con 0x7f2b6810e180 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b67fff640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002c430 tx=0x7f2b600336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b64ff9640 1 -- 172.21.15.49:0/212284871 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60043020 con 0x7f2b6810e180 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6819f450 con 0x7f2b6810e180 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b64ff9640 1 -- 172.21.15.49:0/212284871 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6002ca80 con 0x7f2b6810e180 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b64ff9640 1 -- 172.21.15.49:0/212284871 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6003e450 con 0x7f2b6810e180 2024-03-16T08:18:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.466+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681a6d40 con 0x7f2b6810e180 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.468+0000 7f2b64ff9640 1 -- 172.21.15.49:0/212284871 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f2b60046020 con 0x7f2b6810e180 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 Processor -- start 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 -- start start 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b64ff9640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 0x7f2b540a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8aed30 0x55b33e8b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b33e809490 con 0x55b33e8aed30 2024-03-16T08:18:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b33e7b1c90 con 0x55b33e8a6150 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b70964740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b33e805530 con 0x55b33e8a5830 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b64ff9640 1 -- 172.21.15.49:0/212284871 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2b600f1450 con 0x7f2b6810e180 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 0x7f2b540a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41662/0 (socket says 172.21.15.49:41662) 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 -- 172.21.15.49:0/3231789740 learned_addr learned my addr 172.21.15.49:0/3231789740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b67fff640 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8aed30 0x55b33e8b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 -- 172.21.15.49:0/3231789740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 msgr2=0x55b33e8ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 -- 172.21.15.49:0/3231789740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8aed30 msgr2=0x55b33e8b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8aed30 0x55b33e8b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.469+0000 7f2b677fe640 1 -- 172.21.15.49:0/3231789740 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33e86c500 con 0x55b33e8a5830 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.470+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 0x7f2b540a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009fd0 tx=0x7f2b50013c00 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.470+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009e10 tx=0x7f2b58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b1d4d48150636fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.470+0000 7f2b5dffb640 1 -- 172.21.15.49:0/3231789740 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58019620 con 0x55b33e8a5830 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.470+0000 7f2b5dffb640 1 -- 172.21.15.49:0/3231789740 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b58019ca0 con 0x55b33e8a5830 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.470+0000 7f2b5dffb640 1 -- 172.21.15.49:0/3231789740 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58018da0 con 0x55b33e8a5830 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- 172.21.15.49:0/3231789740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 msgr2=0x55b33e8a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009e10 tx=0x7f2b58011810 comp rx=0 tx=0).stop 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- 172.21.15.49:0/3231789740 shutdown_connections 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8aed30 0x55b33e8b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 --2- 172.21.15.49:0/3231789740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8a5c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- 172.21.15.49:0/3231789740 >> 172.21.15.49:0/3231789740 conn(0x55b33e89ff60 msgr2=0x55b33e8a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- 172.21.15.49:0/3231789740 shutdown_connections 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- 172.21.15.49:0/3231789740 wait complete. 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 -- start start 2024-03-16T08:18:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.471+0000 7f2b70964740 1 Processor -- start 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a6150 0x55b33e8ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b33e8b1b20 con 0x55b33e8a6150 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b33e7c5230 con 0x55b33e8aed30 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b33e8b1ca0 con 0x55b33e8a5830 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 10 client.0 _ll_register_callbacks cb 0x55b33e801010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a6150 0x55b33e8ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:41668/0 (socket says 172.21.15.49:41668) 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 -- 172.21.15.49:0/3585476309 learned_addr learned my addr 172.21.15.49:0/3585476309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b67fff640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 -- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 msgr2=0x55b33e8ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 -- 172.21.15.49:0/3585476309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a6150 msgr2=0x55b33e8ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a6150 0x55b33e8ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 -- 172.21.15.49:0/3585476309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33e7d4100 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b677fe640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009b90 tx=0x7f2b58005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c16d414852000db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58026070 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2b58005250 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.472+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b58019ca0 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2b58032050 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 10 client.25858 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b33e51d3f0 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b58019ca0 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2b580047e0 con 0x55b33e8a5830 2024-03-16T08:18:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 10 client.25858 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 10 client.25858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 20 client.25858 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e8bef60 0x55b33e8c1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 4 client.25858 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33e8169d0 con 0x55b33e8bef60 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.473+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e8bef60 0x55b33e8c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b46ffd640 10 client.25858 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e8bef60 0x55b33e8c1350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94316b80208dd2 server_cookie=ce6bb6b4c9a7e76 in_seq=0 out_seq=0 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b46ffd640 1 -- 172.21.15.49:0/3585476309 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b33e8169d0 con 0x55b33e8bef60 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b46ffd640 10 client.25858 handle_command_reply: tid=0 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b70964740 1 client.25858 shutdown 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 msgr2=0x55b33e8abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.474+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009b90 tx=0x7f2b58005cc0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 shutdown_connections 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e8bef60 0x55b33e8c1350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a6150 0x55b33e8ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b46ffd640 0 client.25858 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 --2- 172.21.15.49:0/3585476309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a5830 0x55b33e8abcc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 >> 172.21.15.49:0/3585476309 conn(0x55b33e89ff60 msgr2=0x55b33e8a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 shutdown_connections 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 1 -- 172.21.15.49:0/3585476309 wait complete. 2024-03-16T08:18:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.475+0000 7f2b70964740 20 client.25858 trim_cache size 0 max 16384 2024-03-16T08:18:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 Processor -- start 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 -- start start 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a6150 0x55b33e8ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681b1e30 con 0x55b33e8a5830 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681af700 con 0x55b33e8aed30 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.488+0000 7f2b70964740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681ad530 con 0x55b33e8a6150 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a6150 0x55b33e8ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40176/0 (socket says 172.21.15.49:40176) 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 -- 172.21.15.49:0/752743886 learned_addr learned my addr 172.21.15.49:0/752743886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 -- 172.21.15.49:0/752743886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a6150 msgr2=0x55b33e8ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a6150 0x55b33e8ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 -- 172.21.15.49:0/752743886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 msgr2=0x55b33e8ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 -- 172.21.15.49:0/752743886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33e5e77d0 con 0x55b33e8aed30 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b67fff640 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b60033ff0 tx=0x7f2b600bd850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f423145487c01 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b5d7fa640 1 -- 172.21.15.49:0/752743886 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600bd9c0 con 0x55b33e8aed30 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b5d7fa640 1 -- 172.21.15.49:0/752743886 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b600c1b10 con 0x55b33e8aed30 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.489+0000 7f2b5d7fa640 1 -- 172.21.15.49:0/752743886 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600c5f30 con 0x55b33e8aed30 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 -- 172.21.15.49:0/752743886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 msgr2=0x55b33e8aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b60033ff0 tx=0x7f2b600bd850 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 -- 172.21.15.49:0/752743886 shutdown_connections 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8aed30 0x55b33e8aae20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8a6150 0x55b33e8ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 --2- 172.21.15.49:0/752743886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 -- 172.21.15.49:0/752743886 >> 172.21.15.49:0/752743886 conn(0x55b33e89ff60 msgr2=0x55b33e89e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 -- 172.21.15.49:0/752743886 shutdown_connections 2024-03-16T08:18:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.490+0000 7f2b70964740 1 -- 172.21.15.49:0/752743886 wait complete. 2024-03-16T08:18:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 -- start start 2024-03-16T08:18:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 Processor -- start 2024-03-16T08:18:42.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 0x55b33e8b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681adbc0 con 0x55b33e8a5830 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681af030 con 0x55b33e8a6150 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681ae290 con 0x55b33e8aed30 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b70964740 10 client.0 _ll_register_callbacks cb 0x7f2b60009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39352/0 (socket says 172.21.15.49:39352) 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b677fe640 1 -- 172.21.15.49:0/4150864380 learned_addr learned my addr 172.21.15.49:0/4150864380 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40184/0 (socket says 172.21.15.49:40184) 2024-03-16T08:18:42.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.491+0000 7f2b67fff640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 0x55b33e8b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 -- 172.21.15.49:0/4150864380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 msgr2=0x55b33e8b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 0x55b33e8b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 -- 172.21.15.49:0/4150864380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 msgr2=0x55b33e8b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 -- 172.21.15.49:0/4150864380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33e8b2d60 con 0x55b33e8a6150 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b67fff640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 0x55b33e8b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b677fe640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55b33e8ab520 tx=0x7f2b5001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc46e047f146a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50014e60 con 0x55b33e8a6150 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2b50005020 con 0x55b33e8a6150 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b600c1b10 con 0x55b33e8a6150 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2b50005220 con 0x55b33e8a6150 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b70964740 10 client.25868 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b33e924f20 con 0x55b33e8a6150 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.492+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b600c1b10 con 0x55b33e8a6150 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2b5001f070 con 0x55b33e8a6150 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 10 client.25868 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 10 client.25868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 20 client.25868 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e925650 0x55b33e927a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 4 client.25868 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b33e810d20 con 0x55b33e925650 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b677fe640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e925650 0x55b33e927a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b46ffd640 10 client.25868 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b677fe640 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e925650 0x55b33e927a70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c111edabfbe0f48 server_cookie=b974fe637a013eaf in_seq=0 out_seq=0 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b46ffd640 1 -- 172.21.15.49:0/4150864380 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b33e810d20 con 0x55b33e925650 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b46ffd640 10 client.25868 handle_command_reply: tid=0 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.493+0000 7f2b70964740 1 client.25868 shutdown 2024-03-16T08:18:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 msgr2=0x55b33e8b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55b33e8ab520 tx=0x7f2b5001b000 comp rx=0 tx=0).stop 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 shutdown_connections 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55b33e925650 0x55b33e927a70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b33e8aed30 0x55b33e8b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b46ffd640 0 client.25868 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b33e8a6150 0x55b33e8b22e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 --2- 172.21.15.49:0/4150864380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b33e8a5830 0x55b33e8b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 >> 172.21.15.49:0/4150864380 conn(0x55b33e89ff60 msgr2=0x55b33e8ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 shutdown_connections 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 1 -- 172.21.15.49:0/4150864380 wait complete. 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b70964740 20 client.25868 trim_cache size 0 max 16384 2024-03-16T08:18:42.501 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 msgr2=0x7f2b540a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 0x7f2b540a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009fd0 tx=0x7f2b50013c00 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 msgr2=0x7f2b6819ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.494+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002c430 tx=0x7f2b600336b0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 shutdown_connections 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b540a2a90 0x7f2b540a4f50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b6810e180 0x7f2b6819ea90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b680692e0 0x7f2b6819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 --2- 172.21.15.49:0/212284871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b680689c0 0x7f2b6819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 >> 172.21.15.49:0/212284871 conn(0x7f2b680777e0 msgr2=0x7f2b6810f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 shutdown_connections 2024-03-16T08:18:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.495+0000 7f2b6dc9a640 1 -- 172.21.15.49:0/212284871 wait complete. 2024-03-16T08:18:42.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 -- 172.21.15.49:0/46268281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 msgr2=0x7f7f4c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 --2- 172.21.15.49:0/46268281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c107b60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009930 tx=0x7f7f4802bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 -- 172.21.15.49:0/46268281 shutdown_connections 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 --2- 172.21.15.49:0/46268281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c102130 0x7f7f4c10e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 --2- 172.21.15.49:0/46268281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c101770 0x7f7f4c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 --2- 172.21.15.49:0/46268281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c107b60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 -- 172.21.15.49:0/46268281 >> 172.21.15.49:0/46268281 conn(0x7f7f4c078ba0 msgr2=0x7f7f4c078fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 -- 172.21.15.49:0/46268281 shutdown_connections 2024-03-16T08:18:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.622+0000 7f7f539ba640 1 -- 172.21.15.49:0/46268281 wait complete. 2024-03-16T08:18:42.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.623+0000 7f7f539ba640 1 Processor -- start 2024-03-16T08:18:42.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.623+0000 7f7f539ba640 1 -- start start 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 0x7f7f4c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c111b50 con 0x7f7f4c107780 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1119d0 con 0x7f7f4c101770 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f539ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c111cd0 con 0x7f7f4c102130 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51f57640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58144/0 (socket says 172.21.15.49:58144) 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 -- 172.21.15.49:0/1276440078 learned_addr learned my addr 172.21.15.49:0/1276440078 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f50f55640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 0x7f7f4c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 -- 172.21.15.49:0/1276440078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 msgr2=0x7f7f4c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 0x7f7f4c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 -- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 msgr2=0x7f7f4c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 -- 172.21.15.49:0/1276440078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19d080 con 0x7f7f4c101770 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f50f55640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 0x7f7f4c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.624+0000 7f7f51756640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009230 tx=0x7f7f4802cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.625+0000 7f7f427fc640 1 -- 172.21.15.49:0/1276440078 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48041070 con 0x7f7f4c101770 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.625+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19d310 con 0x7f7f4c101770 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.625+0000 7f7f427fc640 1 -- 172.21.15.49:0/1276440078 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f48004450 con 0x7f7f4c101770 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.625+0000 7f7f427fc640 1 -- 172.21.15.49:0/1276440078 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48003de0 con 0x7f7f4c101770 2024-03-16T08:18:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.625+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a4c00 con 0x7f7f4c101770 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.627+0000 7f7f5659d740 1 Processor -- start 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.627+0000 7f7f5659d740 1 -- start start 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e46fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e470150 0x55605e4787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55605e3d3490 con 0x55605e478d30 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55605e37bc90 con 0x55605e470150 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f5659d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55605e3cf530 con 0x55605e46f830 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f51f57640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e46fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40476/0 (socket says 172.21.15.49:40476) 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 -- 172.21.15.49:0/3545784863 learned_addr learned my addr 172.21.15.49:0/3545784863 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 -- 172.21.15.49:0/3545784863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 msgr2=0x55605e46fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e46fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 -- 172.21.15.49:0/3545784863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e470150 msgr2=0x55605e4787f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e470150 0x55605e4787f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 -- 172.21.15.49:0/3545784863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605e436500 con 0x55605e478d30 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f51f57640 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e46fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f427fc640 1 -- 172.21.15.49:0/1276440078 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f7f48044070 con 0x7f7f4c101770 2024-03-16T08:18:42.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0114f0 tx=0x7f7f3c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40be42b41322ad0a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.628+0000 7f7f377fe640 1 -- 172.21.15.49:0/3545784863 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01e070 con 0x55605e478d30 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f377fe640 1 -- 172.21.15.49:0/3545784863 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c0035f0 con 0x55605e478d30 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f377fe640 1 -- 172.21.15.49:0/3545784863 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00f820 con 0x55605e478d30 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f427fc640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 0x7f7f380a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f51f57640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 0x7f7f380a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f427fc640 1 -- 172.21.15.49:0/1276440078 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f480f0900 con 0x7f7f4c101770 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f5659d740 1 -- 172.21.15.49:0/3545784863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 msgr2=0x55605e47b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.629+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0114f0 tx=0x7f7f3c0119b0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 -- 172.21.15.49:0/3545784863 shutdown_connections 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e478d30 0x55605e47b120 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e470150 0x55605e4787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3545784863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e46fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 -- 172.21.15.49:0/3545784863 >> 172.21.15.49:0/3545784863 conn(0x55605e469f60 msgr2=0x55605e46a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 -- 172.21.15.49:0/3545784863 shutdown_connections 2024-03-16T08:18:42.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f51f57640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 0x7f7f380a4f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 -- 172.21.15.49:0/3545784863 wait complete. 2024-03-16T08:18:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 -- start start 2024-03-16T08:18:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.630+0000 7f7f5659d740 1 Processor -- start 2024-03-16T08:18:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e476310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e476850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55605e47bb20 con 0x55605e46f830 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55605e38f230 con 0x55605e478d30 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55605e47bca0 con 0x55605e470150 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f5659d740 10 client.0 _ll_register_callbacks cb 0x55605e3cb010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40488/0 (socket says 172.21.15.49:40488) 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 -- 172.21.15.49:0/3809899898 learned_addr learned my addr 172.21.15.49:0/3809899898 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51f57640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e476850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e476310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 -- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 msgr2=0x55605e476310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e476310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 -- 172.21.15.49:0/3809899898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 msgr2=0x55605e476850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e476850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.631+0000 7f7f51756640 1 -- 172.21.15.49:0/3809899898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605e39e100 con 0x55605e46f830 2024-03-16T08:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f51756640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009430 tx=0x7f7f480b91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0965ccf5d002ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4804b040 con 0x55605e46f830 2024-03-16T08:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f480c5380 con 0x55605e46f830 2024-03-16T08:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c0035f0 con 0x55605e46f830 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f480d9020 con 0x55605e46f830 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 10 client.15996 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55605e0e73f0 con 0x55605e46f830 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c0035f0 con 0x55605e46f830 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7f480ee200 con 0x55605e46f830 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 10 client.15996 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 20 client.15996 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e48b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 4 client.15996 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.632+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55605e3e09d0 con 0x55605e489150 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.633+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e48b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.633+0000 7f7f50f55640 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e48b560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=560c0b59094dc22 server_cookie=c494625d413d6003 in_seq=0 out_seq=0 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.633+0000 7f7f34ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f34ff9640 1 -- 172.21.15.49:0/3809899898 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55605e3e09d0 con 0x55605e489150 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f34ff9640 10 client.15996 handle_command_reply: tid=0 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 client.15996 shutdown 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 msgr2=0x55605e475dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009430 tx=0x7f7f480b91d0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 shutdown_connections 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e48b560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e476850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e476310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f34ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 --2- 172.21.15.49:0/3809899898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e475dd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 >> 172.21.15.49:0/3809899898 conn(0x55605e469f60 msgr2=0x55605e46bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 shutdown_connections 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 1 -- 172.21.15.49:0/3809899898 wait complete. 2024-03-16T08:18:42.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.634+0000 7f7f5659d740 20 client.15996 trim_cache size 0 max 16384 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.647+0000 7f7f5659d740 1 Processor -- start 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 -- start start 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e478500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e490dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1afc10 con 0x55605e46f830 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad4e0 con 0x55605e478d30 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f5659d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ab310 con 0x55605e470150 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f51756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e478500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f51f57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e490dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42550/0 (socket says 172.21.15.49:42550) 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f51f57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e490dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58150/0 (socket says 172.21.15.49:58150) 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 -- 172.21.15.49:0/4014568380 learned_addr learned my addr 172.21.15.49:0/4014568380 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 -- 172.21.15.49:0/4014568380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 msgr2=0x55605e490dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e490dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 -- 172.21.15.49:0/4014568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 msgr2=0x55605e478500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e478500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 -- 172.21.15.49:0/4014568380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605e1b17d0 con 0x55605e470150 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.648+0000 7f7f50f55640 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c0037d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34bba60ac4365992 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f36ffd640 1 -- 172.21.15.49:0/4014568380 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c030070 con 0x55605e470150 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f36ffd640 1 -- 172.21.15.49:0/4014568380 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c02a030 con 0x55605e470150 2024-03-16T08:18:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f36ffd640 1 -- 172.21.15.49:0/4014568380 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01e040 con 0x55605e470150 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 -- 172.21.15.49:0/4014568380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 msgr2=0x55605e4848c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c0037d0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 -- 172.21.15.49:0/4014568380 shutdown_connections 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e490dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e470150 0x55605e4848c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 --2- 172.21.15.49:0/4014568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e46f830 0x55605e478500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.649+0000 7f7f5659d740 1 -- 172.21.15.49:0/4014568380 >> 172.21.15.49:0/4014568380 conn(0x55605e469f60 msgr2=0x55605e471150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- 172.21.15.49:0/4014568380 shutdown_connections 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- 172.21.15.49:0/4014568380 wait complete. 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- start start 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 Processor -- start 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e47bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ab9a0 con 0x55605e470150 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ace10 con 0x55605e478d30 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ac070 con 0x55605e46f830 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f5659d740 10 client.0 _ll_register_callbacks cb 0x7f7f48005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f51756640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f51f57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e47bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f51756640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42552/0 (socket says 172.21.15.49:42552) 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f51756640 1 -- 172.21.15.49:0/331157939 learned_addr learned my addr 172.21.15.49:0/331157939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.650+0000 7f7f50f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40504/0 (socket says 172.21.15.49:40504) 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 -- 172.21.15.49:0/331157939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 msgr2=0x55605e47bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e47bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 -- 172.21.15.49:0/331157939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 msgr2=0x55605e480fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 -- 172.21.15.49:0/331157939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605e47c600 con 0x55605e46f830 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f50f55640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f51756640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f480f0ba0 tx=0x7f7f480bc4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d9c6fd35ca599e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4804b040 con 0x55605e46f830 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f480b8bc0 con 0x55605e46f830 2024-03-16T08:18:42.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c02a030 con 0x55605e46f830 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f480bd570 con 0x55605e46f830 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f5659d740 10 client.25882 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.651+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55605e47c600 con 0x55605e46f830 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c02a030 con 0x55605e46f830 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7f480a4070 con 0x55605e46f830 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 10 client.25882 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 10 client.25882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 20 client.25882 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e4f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 4 client.25882 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55605e3dad20 con 0x55605e489150 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f50f55640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e4f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f34ff9640 10 client.25882 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:42.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.652+0000 7f7f50f55640 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e4f11b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f8d9b4815676baa server_cookie=bc789c7032ed9889 in_seq=0 out_seq=0 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f34ff9640 1 -- 172.21.15.49:0/331157939 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55605e3dad20 con 0x55605e489150 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f34ff9640 10 client.25882 handle_command_reply: tid=0 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 client.25882 shutdown 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 msgr2=0x55605e480a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f480f0ba0 tx=0x7f7f480bc4f0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 shutdown_connections 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55605e489150 0x55605e4f11b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55605e478d30 0x55605e47bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f34ff9640 0 client.25882 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55605e470150 0x55605e480fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 --2- 172.21.15.49:0/331157939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55605e46f830 0x55605e480a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 >> 172.21.15.49:0/331157939 conn(0x55605e469f60 msgr2=0x55605e475bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 shutdown_connections 2024-03-16T08:18:42.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 1 -- 172.21.15.49:0/331157939 wait complete. 2024-03-16T08:18:42.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f5659d740 20 client.25882 trim_cache size 0 max 16384 2024-03-16T08:18:42.661 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 msgr2=0x7f7f380a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 0x7f7f380a4f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44009500 comp rx=0 tx=0).stop 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 msgr2=0x7f7f4c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.653+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009230 tx=0x7f7f4802cb30 comp rx=0 tx=0).stop 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 shutdown_connections 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f380a2a90 0x7f7f380a4f50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f4c107780 0x7f7f4c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c102130 0x7f7f4c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 --2- 172.21.15.49:0/1276440078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f4c101770 0x7f7f4c198090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 >> 172.21.15.49:0/1276440078 conn(0x7f7f4c078ba0 msgr2=0x7f7f4c1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 shutdown_connections 2024-03-16T08:18:42.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.654+0000 7f7f539ba640 1 -- 172.21.15.49:0/1276440078 wait complete. 2024-03-16T08:18:42.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:18:42.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 -- 172.21.15.49:0/2699214183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1006a0 msgr2=0x7f03cc100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/2699214183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1006a0 0x7f03cc100b00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009900 tx=0x7f03bc02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 -- 172.21.15.49:0/2699214183 shutdown_connections 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/2699214183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc101040 0x7f03cc10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/2699214183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1006a0 0x7f03cc100b00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/2699214183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc105580 0x7f03cc105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 -- 172.21.15.49:0/2699214183 >> 172.21.15.49:0/2699214183 conn(0x7f03cc077770 msgr2=0x7f03cc077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 -- 172.21.15.49:0/2699214183 shutdown_connections 2024-03-16T08:18:42.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.782+0000 7f03d3aa1640 1 -- 172.21.15.49:0/2699214183 wait complete. 2024-03-16T08:18:42.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 Processor -- start 2024-03-16T08:18:42.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 -- start start 2024-03-16T08:18:42.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 0x7f03cc198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 0x7f03cc1a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc110ac0 con 0x7f03cc101040 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc110940 con 0x7f03cc105580 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d3aa1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc110c40 con 0x7f03cc1006a0 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d183d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 0x7f03cc198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40522/0 (socket says 172.21.15.49:40522) 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d103c640 1 -- 172.21.15.49:0/1570421973 learned_addr learned my addr 172.21.15.49:0/1570421973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.783+0000 7f03d203e640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 0x7f03cc1a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 -- 172.21.15.49:0/1570421973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 msgr2=0x7f03cc198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 0x7f03cc198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 -- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 msgr2=0x7f03cc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 -- 172.21.15.49:0/1570421973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1a4480 con 0x7f03cc105580 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d183d640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 0x7f03cc198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d103c640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d203e640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 0x7f03cc1a3f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ec50 tx=0x7f03c400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03c2ffd640 1 -- 172.21.15.49:0/1570421973 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4003b70 con 0x7f03cc105580 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03c2ffd640 1 -- 172.21.15.49:0/1570421973 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c4003d10 con 0x7f03cc105580 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1a46b0 con 0x7f03cc105580 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03c2ffd640 1 -- 172.21.15.49:0/1570421973 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4010890 con 0x7f03cc105580 2024-03-16T08:18:42.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.784+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1a4bd0 con 0x7f03cc105580 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.786+0000 7f03c2ffd640 1 -- 172.21.15.49:0/1570421973 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f03c4003fc0 con 0x7f03cc105580 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03c2ffd640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 0x7f03b80a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03d6769740 1 Processor -- start 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03c2ffd640 1 -- 172.21.15.49:0/1570421973 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f03c40cf600 con 0x7f03cc105580 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03d183d640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 0x7f03b80a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03d6769740 1 -- start start 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.787+0000 7f03d6769740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4d830 0x565426e4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d6769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d6769740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d6769740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565426db1490 con 0x565426e4d830 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d6769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565426d59c90 con 0x565426e4e150 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d6769740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565426dad530 con 0x565426e56d30 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58190/0 (socket says 172.21.15.49:58190) 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d183d640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 0x7f03b80a4fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009f40 tx=0x7f03c8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 -- 172.21.15.49:0/528241133 learned_addr learned my addr 172.21.15.49:0/528241133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d183d640 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4d830 0x565426e4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d203e640 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 -- 172.21.15.49:0/528241133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 msgr2=0x565426e59120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e59120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 -- 172.21.15.49:0/528241133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4d830 msgr2=0x565426e4dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4d830 0x565426e4dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 -- 172.21.15.49:0/528241133 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565426e14500 con 0x565426e4e150 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03d103c640 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036040 tx=0x7f03bc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37e8106de9b4a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03b7fff640 1 -- 172.21.15.49:0/528241133 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc043070 con 0x565426e4e150 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03b7fff640 1 -- 172.21.15.49:0/528241133 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc004a70 con 0x565426e4e150 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.788+0000 7f03b7fff640 1 -- 172.21.15.49:0/528241133 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc005140 con 0x565426e4e150 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 -- 172.21.15.49:0/528241133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 msgr2=0x565426e567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036040 tx=0x7f03bc009560 comp rx=0 tx=0).stop 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 -- 172.21.15.49:0/528241133 shutdown_connections 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e59120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e567f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 --2- 172.21.15.49:0/528241133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4d830 0x565426e4dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 -- 172.21.15.49:0/528241133 >> 172.21.15.49:0/528241133 conn(0x565426e47f60 msgr2=0x565426e48340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 -- 172.21.15.49:0/528241133 shutdown_connections 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.789+0000 7f03d6769740 1 -- 172.21.15.49:0/528241133 wait complete. 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 -- start start 2024-03-16T08:18:42.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 Processor -- start 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565426e59ac0 con 0x565426e4e150 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565426d6d230 con 0x565426e4d830 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565426e59c40 con 0x565426e56d30 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d6769740 10 client.0 _ll_register_callbacks cb 0x565426da9010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d203e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d183d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40548/0 (socket says 172.21.15.49:40548) 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d203e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42594/0 (socket says 172.21.15.49:42594) 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d103c640 1 -- 172.21.15.49:0/1972807097 learned_addr learned my addr 172.21.15.49:0/1972807097 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.790+0000 7f03d183d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58196/0 (socket says 172.21.15.49:58196) 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 -- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 msgr2=0x565426e53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 -- 172.21.15.49:0/1972807097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 msgr2=0x565426e542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 -- 172.21.15.49:0/1972807097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565426d7c100 con 0x565426e56d30 2024-03-16T08:18:42.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d103c640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d183d640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d203e640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ccbf0 tx=0x7f03c409b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a189f4fe4dad7e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c409c160 con 0x565426e56d30 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f03c40ccf00 con 0x565426e56d30 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc004a70 con 0x565426e56d30 2024-03-16T08:18:42.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f03c40b3070 con 0x565426e56d30 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d6769740 10 client.25894 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565426ac53f0 con 0x565426e56d30 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.791+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc004a70 con 0x565426e56d30 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f03c4004b40 con 0x565426e56d30 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 10 client.25894 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 20 client.25894 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426e670a0 0x565426e694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 4 client.25894 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:42.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565426dbe9d0 con 0x565426e670a0 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d183d640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426e670a0 0x565426e694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03d183d640 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426e670a0 0x565426e694d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28439e83bda9c91c server_cookie=c32b9d213c06a739 in_seq=0 out_seq=0 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.792+0000 7f03b57fa640 10 client.25894 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03b57fa640 1 -- 172.21.15.49:0/1972807097 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565426dbe9d0 con 0x565426e670a0 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03b57fa640 10 client.25894 handle_command_reply: tid=0 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 client.25894 shutdown 2024-03-16T08:18:42.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 msgr2=0x565426e54820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ccbf0 tx=0x7f03c409b8f0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 shutdown_connections 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426e670a0 0x565426e694d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e56d30 0x565426e54820 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 --2- 172.21.15.49:0/1972807097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4d830 0x565426e53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03b57fa640 0 client.25894 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 >> 172.21.15.49:0/1972807097 conn(0x565426e47f60 msgr2=0x565426e49cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 shutdown_connections 2024-03-16T08:18:42.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 1 -- 172.21.15.49:0/1972807097 wait complete. 2024-03-16T08:18:42.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.793+0000 7f03d6769740 20 client.25894 trim_cache size 0 max 16384 2024-03-16T08:18:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 Processor -- start 2024-03-16T08:18:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 -- start start 2024-03-16T08:18:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 0x565426e52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1afb90 con 0x565426e56d30 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1ad460 con 0x565426e4e150 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.807+0000 7f03d6769740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1ab290 con 0x565426e4d830 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d203e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 0x565426e52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58208/0 (socket says 172.21.15.49:58208) 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 -- 172.21.15.49:0/1291238859 learned_addr learned my addr 172.21.15.49:0/1291238859 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d183d640 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 -- 172.21.15.49:0/1291238859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 msgr2=0x565426e564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 -- 172.21.15.49:0/1291238859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 msgr2=0x565426e52f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 0x565426e52f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 -- 172.21.15.49:0/1291238859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565426e43640 con 0x565426e4e150 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d203e640 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 0x565426e52f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03d103c640 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02c2e0 tx=0x7f03bc02cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b4b75b9e12762f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03b77fe640 1 -- 172.21.15.49:0/1291238859 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc04c070 con 0x565426e4e150 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03b77fe640 1 -- 172.21.15.49:0/1291238859 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc004350 con 0x565426e4e150 2024-03-16T08:18:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.808+0000 7f03b77fe640 1 -- 172.21.15.49:0/1291238859 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc043070 con 0x565426e4e150 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- 172.21.15.49:0/1291238859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 msgr2=0x565426e42e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02c2e0 tx=0x7f03bc02cc40 comp rx=0 tx=0).stop 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- 172.21.15.49:0/1291238859 shutdown_connections 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e56d30 0x565426e52f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e4e150 0x565426e42e70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 --2- 172.21.15.49:0/1291238859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- 172.21.15.49:0/1291238859 >> 172.21.15.49:0/1291238859 conn(0x565426e47f60 msgr2=0x565426e53d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- 172.21.15.49:0/1291238859 shutdown_connections 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- 172.21.15.49:0/1291238859 wait complete. 2024-03-16T08:18:42.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.809+0000 7f03d6769740 1 -- start start 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 Processor -- start 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e5a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 0x565426ecb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1ab920 con 0x565426e4e150 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1acd90 con 0x565426e56d30 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1abff0 con 0x565426e4d830 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d6769740 10 client.0 _ll_register_callbacks cb 0x7f03c40ca0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40560/0 (socket says 172.21.15.49:40560) 2024-03-16T08:18:42.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d203e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 0x565426ecb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 -- 172.21.15.49:0/1185003276 learned_addr learned my addr 172.21.15.49:0/1185003276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d183d640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e5a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 -- 172.21.15.49:0/1185003276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 msgr2=0x565426e5a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e5a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 -- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 msgr2=0x565426ecb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 0x565426ecb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d103c640 1 -- 172.21.15.49:0/1185003276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565426b8f7d0 con 0x565426e4e150 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d203e640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 0x565426ecb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.810+0000 7f03d183d640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e5a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d103c640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03bc004500 tx=0x7f03bc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50e62aeb2aada7f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc04c040 con 0x565426e4e150 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc004350 con 0x565426e4e150 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f03bc005140 con 0x565426e4e150 2024-03-16T08:18:42.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f03bc043040 con 0x565426e4e150 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 10 client.16020 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565426ecca30 con 0x565426e4e150 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc004350 con 0x565426e4e150 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f03bc063630 con 0x565426e4e150 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 10 client.16020 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 20 client.16020 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426eccd00 0x565426ecf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 4 client.16020 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.811+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565426db8d20 con 0x565426eccd00 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03d183d640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426eccd00 0x565426ecf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03b57fa640 10 client.16020 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:42.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03d183d640 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426eccd00 0x565426ecf0f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64fc208d358b3f96 server_cookie=9dbcf144c8167a4a in_seq=0 out_seq=0 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03b57fa640 1 -- 172.21.15.49:0/1185003276 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565426db8d20 con 0x565426eccd00 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03b57fa640 10 client.16020 handle_command_reply: tid=0 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03d6769740 1 client.16020 shutdown 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 msgr2=0x565426e5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.812+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03bc004500 tx=0x7f03bc009680 comp rx=0 tx=0).stop 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 shutdown_connections 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565426eccd00 0x565426ecf0f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565426e56d30 0x565426ecb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565426e4e150 0x565426e5a580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 --2- 172.21.15.49:0/1185003276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565426e4d830 0x565426e5a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03b57fa640 0 client.16020 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 >> 172.21.15.49:0/1185003276 conn(0x565426e47f60 msgr2=0x565426e492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 shutdown_connections 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 1 -- 172.21.15.49:0/1185003276 wait complete. 2024-03-16T08:18:42.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d6769740 20 client.16020 trim_cache size 0 max 16384 2024-03-16T08:18:42.820 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:18:42.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 msgr2=0x7f03b80a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 0x7f03b80a4fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009f40 tx=0x7f03c8005f90 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 msgr2=0x7f03cc1a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 0x7f03cc1a3f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ec50 tx=0x7f03c400c6a0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 shutdown_connections 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f03b80a2b10 0x7f03b80a4fd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03cc105580 0x7f03cc1a3f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc101040 0x7f03cc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 --2- 172.21.15.49:0/1570421973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03cc1006a0 0x7f03cc198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 >> 172.21.15.49:0/1570421973 conn(0x7f03cc077770 msgr2=0x7f03cc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 shutdown_connections 2024-03-16T08:18:42.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:42.813+0000 7f03d3aa1640 1 -- 172.21.15.49:0/1570421973 wait complete. 2024-03-16T08:18:43.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:42 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1119981324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:43.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:42 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1119981324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:43.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:42 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1119981324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:18:43.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:18:43.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:18:43.827 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- 172.21.15.49:0/1210413730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208100650 msgr2=0x7fa208100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1210413730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208100650 0x7fa208100ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009890 tx=0x7fa1f802bb70 comp rx=0 tx=0).stop 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- 172.21.15.49:0/1210413730 shutdown_connections 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1210413730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208101010 0x7fa20810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1210413730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208100650 0x7fa208100ad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1210413730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208106660 0x7fa208106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- 172.21.15.49:0/1210413730 >> 172.21.15.49:0/1210413730 conn(0x7fa2080777e0 msgr2=0x7fa208077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- 172.21.15.49:0/1210413730 shutdown_connections 2024-03-16T08:18:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- 172.21.15.49:0/1210413730 wait complete. 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 Processor -- start 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 -- start start 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208100650 0x7fa2081980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.951+0000 7fa20eae9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208106660 0x7fa20819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20eae9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa208110a70 con 0x7fa208101010 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20eae9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081108f0 con 0x7fa208100650 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20eae9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa208110bf0 con 0x7fa208106660 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40582/0 (socket says 172.21.15.49:40582) 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 -- 172.21.15.49:0/1862872167 learned_addr learned my addr 172.21.15.49:0/1862872167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20d086640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208106660 0x7fa20819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20c885640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208100650 0x7fa2081980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 -- 172.21.15.49:0/1862872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208106660 msgr2=0x7fa20819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208106660 0x7fa20819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 -- 172.21.15.49:0/1862872167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208100650 msgr2=0x7fa2081980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208100650 0x7fa2081980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 -- 172.21.15.49:0/1862872167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20819d0b0 con 0x7fa208101010 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80057c0 tx=0x7fa1f80045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fdffb640 1 -- 172.21.15.49:0/1862872167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8041020 con 0x7fa208101010 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20819d340 con 0x7fa208101010 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fdffb640 1 -- 172.21.15.49:0/1862872167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f802c9f0 con 0x7fa208101010 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.952+0000 7fa1fdffb640 1 -- 172.21.15.49:0/1862872167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8039040 con 0x7fa208101010 2024-03-16T08:18:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.953+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a4c30 con 0x7fa208101010 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa1fdffb640 1 -- 172.21.15.49:0/1862872167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa1f8044020 con 0x7fa208101010 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 Processor -- start 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 -- start start 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f95460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318fa1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565318ef90d0 con 0x565318f96710 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565318ea1c70 con 0x565318f9eda0 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa2117a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565318ef5170 con 0x565318f95a30 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa1fdffb640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 0x7fa1f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa20d086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f95460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318fa1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa20c885640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa20c885640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42656/0 (socket says 172.21.15.49:42656) 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa20c885640 1 -- 172.21.15.49:0/986554919 learned_addr learned my addr 172.21.15.49:0/986554919 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.956+0000 7fa20d086640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 0x7fa1f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa1fdffb640 1 -- 172.21.15.49:0/1862872167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa1f80f04f0 con 0x7fa208101010 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 -- 172.21.15.49:0/986554919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 msgr2=0x565318fa1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318fa1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 -- 172.21.15.49:0/986554919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 msgr2=0x565318f95460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f95460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 -- 172.21.15.49:0/986554919 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565318cd57e0 con 0x565318f95a30 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20c885640 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa204009c60 tx=0x7fa204011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d20eabba3c4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa20d086640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 0x7fa1f00a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa200005c40 tx=0x7fa200009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa1eeffd640 1 -- 172.21.15.49:0/986554919 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204005020 con 0x565318f95a30 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa1eeffd640 1 -- 172.21.15.49:0/986554919 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2040047a0 con 0x565318f95a30 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.957+0000 7fa1eeffd640 1 -- 172.21.15.49:0/986554919 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204019810 con 0x565318f95a30 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- 172.21.15.49:0/986554919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 msgr2=0x565318f95eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa204009c60 tx=0x7fa204011810 comp rx=0 tx=0).stop 2024-03-16T08:18:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- 172.21.15.49:0/986554919 shutdown_connections 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318fa1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f95a30 0x565318f95eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 --2- 172.21.15.49:0/986554919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f95460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- 172.21.15.49:0/986554919 >> 172.21.15.49:0/986554919 conn(0x565318f5cc50 msgr2=0x565318f90480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- 172.21.15.49:0/986554919 shutdown_connections 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- 172.21.15.49:0/986554919 wait complete. 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 -- start start 2024-03-16T08:18:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.958+0000 7fa2117a8740 1 Processor -- start 2024-03-16T08:18:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f96710 0x565318f9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318f9c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565318fa1c10 con 0x565318f95a30 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565318eb50d0 con 0x565318f9eda0 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565318fa1d90 con 0x565318f96710 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa2117a8740 10 client.0 _ll_register_callbacks cb 0x565318e7e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40594/0 (socket says 172.21.15.49:40594) 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 -- 172.21.15.49:0/1837420080 learned_addr learned my addr 172.21.15.49:0/1837420080 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20d086640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318f9c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f96710 0x565318f9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 -- 172.21.15.49:0/1837420080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f96710 msgr2=0x565318f9c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f96710 0x565318f9c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 -- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 msgr2=0x565318f9c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318f9c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 -- 172.21.15.49:0/1837420080 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565318d35f70 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa20c885640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa20401b040 tx=0x7fa204005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0ed81794e0d880 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.959+0000 7fa1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040053d0 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa204019510 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2040047a0 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa204031020 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 10 client.16032 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565318bcd170 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2040047a0 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa204036550 con 0x565318f95a30 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 10 client.16032 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 20 client.16032 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565318faed30 0x565318fb1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 4 client.16032 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.960+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565318f06610 con 0x565318faed30 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.961+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565318faed30 0x565318fb1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.961+0000 7fa1e7fff640 10 client.16032 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.961+0000 7fa1fffff640 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565318faed30 0x565318fb1120 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64dcef3e364b91a0 server_cookie=8948ca896eb1409c in_seq=0 out_seq=0 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa1e7fff640 1 -- 172.21.15.49:0/1837420080 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565318f06610 con 0x565318faed30 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa1e7fff640 10 client.16032 handle_command_reply: tid=0 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 client.16032 shutdown 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 msgr2=0x565318f9bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa20401b040 tx=0x7fa204005df0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 shutdown_connections 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565318faed30 0x565318fb1120 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f9eda0 0x565318f9c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f96710 0x565318f9c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa1e7fff640 0 client.16032 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 --2- 172.21.15.49:0/1837420080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318f9bd60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 >> 172.21.15.49:0/1837420080 conn(0x565318f5cc50 msgr2=0x565318f921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 shutdown_connections 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.962+0000 7fa2117a8740 1 -- 172.21.15.49:0/1837420080 wait complete. 2024-03-16T08:18:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:43.963+0000 7fa2117a8740 20 client.16032 trim_cache size 0 max 16384 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 Processor -- start 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 -- start start 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565318fb68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081adba0 con 0x565318f96710 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081ab470 con 0x565318f95a30 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.001+0000 7fa2117a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081a92a0 con 0x565318f9eda0 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20d086640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565318fb68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58274/0 (socket says 172.21.15.49:58274) 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40604/0 (socket says 172.21.15.49:40604) 2024-03-16T08:18:44.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 -- 172.21.15.49:0/572919601 learned_addr learned my addr 172.21.15.49:0/572919601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 -- 172.21.15.49:0/572919601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 msgr2=0x565318fb68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565318fb68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 -- 172.21.15.49:0/572919601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 msgr2=0x565318f9b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 -- 172.21.15.49:0/572919601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565318fb16b0 con 0x565318f95a30 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20d086640 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565318fb68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1fffff640 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa20c885640 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa204036a90 tx=0x7fa204009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ec9270c753e5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1ee7fc640 1 -- 172.21.15.49:0/572919601 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204026070 con 0x565318f95a30 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1ee7fc640 1 -- 172.21.15.49:0/572919601 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20403b020 con 0x565318f95a30 2024-03-16T08:18:44.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.002+0000 7fa1ee7fc640 1 -- 172.21.15.49:0/572919601 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204036430 con 0x565318f95a30 2024-03-16T08:18:44.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 -- 172.21.15.49:0/572919601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 msgr2=0x565318faaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa204036a90 tx=0x7fa204009680 comp rx=0 tx=0).stop 2024-03-16T08:18:44.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 -- 172.21.15.49:0/572919601 shutdown_connections 2024-03-16T08:18:44.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565318fb68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f96710 0x565318f9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 --2- 172.21.15.49:0/572919601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f95a30 0x565318faaa20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 -- 172.21.15.49:0/572919601 >> 172.21.15.49:0/572919601 conn(0x565318f5cc50 msgr2=0x565318f9bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 -- 172.21.15.49:0/572919601 shutdown_connections 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.003+0000 7fa2117a8740 1 -- 172.21.15.49:0/572919601 wait complete. 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 -- start start 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 Processor -- start 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 0x565318fa3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565319013f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081a9930 con 0x565318f95a30 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081aada0 con 0x565318f96710 2024-03-16T08:18:44.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081aa000 con 0x565318f9eda0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa2117a8740 10 client.0 _ll_register_callbacks cb 0x7fa1f80bc7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa20c885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa20c885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40606/0 (socket says 172.21.15.49:40606) 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa1fffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 0x565318fa3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa20c885640 1 -- 172.21.15.49:0/272688277 learned_addr learned my addr 172.21.15.49:0/272688277 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.004+0000 7fa20d086640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565319013f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 -- 172.21.15.49:0/272688277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 msgr2=0x565319013f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565319013f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 -- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 msgr2=0x565318fa3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 0x565318fa3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 -- 172.21.15.49:0/272688277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565318f94a60 con 0x565318f95a30 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa1fffff640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 0x565318fa3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa20c885640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa204036a60 tx=0x7fa2040046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343574b4722cd287 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204046060 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa204004c60 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa20403b020 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa204026040 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa2117a8740 10 client.16038 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.005+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565319015530 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20403b020 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2040054a0 con 0x565318f95a30 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 10 client.16038 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:44.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 20 client.16038 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565319015b90 0x565319017f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 4 client.16038 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565318f00960 con 0x565319015b90 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa1fffff640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565319015b90 0x565319017f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa1e7fff640 10 client.16038 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.006+0000 7fa1fffff640 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565319015b90 0x565319017f80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7d06cbd49f1f64 server_cookie=d95104dcc6ebee49 in_seq=0 out_seq=0 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa1e7fff640 1 -- 172.21.15.49:0/272688277 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565318f00960 con 0x565319015b90 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa1e7fff640 10 client.16038 handle_command_reply: tid=0 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 client.16038 shutdown 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 msgr2=0x565318fa2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa204036a60 tx=0x7fa2040046d0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 shutdown_connections 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565319015b90 0x565319017f80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565318f9eda0 0x565319013f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565318f96710 0x565318fa3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 --2- 172.21.15.49:0/272688277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565318f95a30 0x565318fa2fe0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 >> 172.21.15.49:0/272688277 conn(0x565318f5cc50 msgr2=0x565318f8f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa1e7fff640 0 client.16038 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 shutdown_connections 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 1 -- 172.21.15.49:0/272688277 wait complete. 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.008+0000 7fa2117a8740 20 client.16038 trim_cache size 0 max 16384 2024-03-16T08:18:44.014 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:18:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "41cc94ce-e7ed-43c9-878b-b1c15b3ef2ec", 2024-03-16T08:18:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:18:44.015 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:44.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 msgr2=0x7fa1f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 0x7fa1f00a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa200005c40 tx=0x7fa200009500 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 msgr2=0x7fa208198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80057c0 tx=0x7fa1f80045d0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 shutdown_connections 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1f00a2970 0x7fa1f00a4e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa208106660 0x7fa20819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa208101010 0x7fa208198600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 --2- 172.21.15.49:0/1862872167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa208100650 0x7fa2081980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 >> 172.21.15.49:0/1862872167 conn(0x7fa2080777e0 msgr2=0x7fa208068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 shutdown_connections 2024-03-16T08:18:44.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.009+0000 7fa20eae9640 1 -- 172.21.15.49:0/1862872167 wait complete. 2024-03-16T08:18:44.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:18:44.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:18:44.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:43 smithi129 ceph-mon[36561]: pgmap v48: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:44.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:43 smithi154 ceph-mon[36471]: pgmap v48: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.137+0000 7f3b44d22640 1 -- 172.21.15.49:0/3359846777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40101010 msgr2=0x7f3b4010d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.137+0000 7f3b44d22640 1 --2- 172.21.15.49:0/3359846777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40101010 0x7f3b4010d520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009cd0 tx=0x7f3b3802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.137+0000 7f3b44d22640 1 -- 172.21.15.49:0/3359846777 shutdown_connections 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 --2- 172.21.15.49:0/3359846777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40101010 0x7f3b4010d520 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 --2- 172.21.15.49:0/3359846777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 0x7f3b40100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 --2- 172.21.15.49:0/3359846777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40106660 0x7f3b40106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 -- 172.21.15.49:0/3359846777 >> 172.21.15.49:0/3359846777 conn(0x7f3b400777e0 msgr2=0x7f3b40077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 -- 172.21.15.49:0/3359846777 shutdown_connections 2024-03-16T08:18:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 -- 172.21.15.49:0/3359846777 wait complete. 2024-03-16T08:18:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 Processor -- start 2024-03-16T08:18:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.138+0000 7f3b44d22640 1 -- start start 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 0x7f3b40198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40101010 0x7f3b40199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40110a70 con 0x7f3b40106660 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401108f0 con 0x7f3b40100650 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40110bf0 con 0x7f3b40101010 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40614/0 (socket says 172.21.15.49:40614) 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/320770521 learned_addr learned my addr 172.21.15.49:0/320770521 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 0x7f3b40198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b37fff640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40101010 0x7f3b40199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/320770521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40101010 msgr2=0x7f3b40199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40101010 0x7f3b40199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/320770521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 msgr2=0x7f3b40198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 0x7f3b40198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/320770521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401a44b0 con 0x7f3b40106660 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b380348f0 tx=0x7f3b38032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b377fe640 1 -- 172.21.15.49:0/320770521 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38041020 con 0x7f3b40106660 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b377fe640 1 -- 172.21.15.49:0/320770521 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38032e80 con 0x7f3b40106660 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.139+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401a4680 con 0x7f3b40106660 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.140+0000 7f3b377fe640 1 -- 172.21.15.49:0/320770521 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38039040 con 0x7f3b40106660 2024-03-16T08:18:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.140+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401a4ae0 con 0x7f3b40106660 2024-03-16T08:18:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.142+0000 7f3b377fe640 1 -- 172.21.15.49:0/320770521 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3b38044020 con 0x7f3b40106660 2024-03-16T08:18:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.142+0000 7f3b479eb740 1 Processor -- start 2024-03-16T08:18:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.142+0000 7f3b479eb740 1 -- start start 2024-03-16T08:18:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b377fe640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 0x7f3b2c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005528530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005531110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55700548c500 con 0x557005531650 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557005434be0 con 0x557005528a70 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b479eb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5570054885a0 con 0x557005528150 2024-03-16T08:18:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b37fff640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 0x7f3b2c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40626/0 (socket says 172.21.15.49:40626) 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/2696658900 learned_addr learned my addr 172.21.15.49:0/2696658900 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005528530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b37fff640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005531110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b377fe640 1 -- 172.21.15.49:0/320770521 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3b380f08b0 con 0x7f3b40106660 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 -- 172.21.15.49:0/2696658900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 msgr2=0x557005531110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005531110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 -- 172.21.15.49:0/2696658900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 msgr2=0x557005533a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3e59c640 1 -- 172.21.15.49:0/2696658900 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570054484b0 con 0x557005528150 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.143+0000 7f3b37fff640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 0x7f3b2c0a4e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28013460 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005528530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009c00 tx=0x7f3b30005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd2848b08adac7dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b34ff9640 1 -- 172.21.15.49:0/2696658900 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004050 con 0x557005528150 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b34ff9640 1 -- 172.21.15.49:0/2696658900 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005500 con 0x557005528150 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b34ff9640 1 -- 172.21.15.49:0/2696658900 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004c60 con 0x557005528150 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b479eb740 1 -- 172.21.15.49:0/2696658900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 msgr2=0x557005528530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.144+0000 7f3b479eb740 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005528530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009c00 tx=0x7f3b30005850 comp rx=0 tx=0).stop 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- 172.21.15.49:0/2696658900 shutdown_connections 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x557005533a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005531110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- 172.21.15.49:0/2696658900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005528530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- 172.21.15.49:0/2696658900 >> 172.21.15.49:0/2696658900 conn(0x557005522880 msgr2=0x557005522c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- 172.21.15.49:0/2696658900 shutdown_connections 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- 172.21.15.49:0/2696658900 wait complete. 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- start start 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 Processor -- start 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 0x55700552ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 0x55700552f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557005534440 con 0x557005528150 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557005448160 con 0x557005531650 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.145+0000 7f3b479eb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5570055345c0 con 0x557005528a70 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b479eb740 10 client.0 _ll_register_callbacks cb 0x55700542c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40632/0 (socket says 172.21.15.49:40632) 2024-03-16T08:18:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 -- 172.21.15.49:0/3592622782 learned_addr learned my addr 172.21.15.49:0/3592622782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b37fff640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 0x55700552ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 0x55700552f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 -- 172.21.15.49:0/3592622782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 msgr2=0x55700552ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 0x55700552ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 -- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 msgr2=0x55700552f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 0x55700552f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 -- 172.21.15.49:0/3592622782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700526a7d0 con 0x557005528150 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b37fff640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 0x55700552ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 0x55700552f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b30003820 tx=0x7f3b30003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41104930245f582 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003c80 con 0x557005528150 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3b300049e0 con 0x557005528150 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.146+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30005500 con 0x557005528150 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3b30003e20 con 0x557005528150 2024-03-16T08:18:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 10 client.16050 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5570051a03f0 con 0x557005528150 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005500 con 0x557005528150 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3b300195f0 con 0x557005528150 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 10 client.16050 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 20 client.16050 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557005541510 0x557005543900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 4 client.16050 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557005499a40 con 0x557005541510 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.147+0000 7f3b37fff640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557005541510 0x557005543900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.148+0000 7f3b25ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.148+0000 7f3b37fff640 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557005541510 0x557005543900 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db07a5b33a04318c server_cookie=6b79686e3189eeb in_seq=0 out_seq=0 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b25ffb640 1 -- 172.21.15.49:0/3592622782 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557005499a40 con 0x557005541510 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b25ffb640 10 client.16050 handle_command_reply: tid=0 2024-03-16T08:18:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 client.16050 shutdown 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 msgr2=0x55700552e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b30003820 tx=0x7f3b30003850 comp rx=0 tx=0).stop 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 shutdown_connections 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557005541510 0x557005543900 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005531650 0x55700552f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b25ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528a70 0x55700552ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 --2- 172.21.15.49:0/3592622782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005528150 0x55700552e670 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 >> 172.21.15.49:0/3592622782 conn(0x557005522880 msgr2=0x5570055245d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 shutdown_connections 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 1 -- 172.21.15.49:0/3592622782 wait complete. 2024-03-16T08:18:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.149+0000 7f3b479eb740 20 client.16050 trim_cache size 0 max 16384 2024-03-16T08:18:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 Processor -- start 2024-03-16T08:18:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 -- start start 2024-03-16T08:18:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700553d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401afc60 con 0x557005531650 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ad530 con 0x557005528a70 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b479eb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ab380 con 0x557005528150 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700553d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58336/0 (socket says 172.21.15.49:58336) 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42736/0 (socket says 172.21.15.49:42736) 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700553d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40648/0 (socket says 172.21.15.49:40648) 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.180+0000 7f3b37fff640 1 -- 172.21.15.49:0/300330118 learned_addr learned my addr 172.21.15.49:0/300330118 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 -- 172.21.15.49:0/300330118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 msgr2=0x557005530d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 -- 172.21.15.49:0/300330118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 msgr2=0x55700553d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700553d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 -- 172.21.15.49:0/300330118 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570051600f0 con 0x557005528a70 2024-03-16T08:18:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b37fff640 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x557005527c20 tx=0x7f3b28021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a160fc1ac70d21e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b27fff640 1 -- 172.21.15.49:0/300330118 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800fe20 con 0x557005528a70 2024-03-16T08:18:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b27fff640 1 -- 172.21.15.49:0/300330118 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b28010de0 con 0x557005528a70 2024-03-16T08:18:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.181+0000 7f3b27fff640 1 -- 172.21.15.49:0/300330118 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2802d630 con 0x557005528a70 2024-03-16T08:18:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- 172.21.15.49:0/300330118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 msgr2=0x557005528e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x557005527c20 tx=0x7f3b28021000 comp rx=0 tx=0).stop 2024-03-16T08:18:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- 172.21.15.49:0/300330118 shutdown_connections 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700553d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x557005528e50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- 172.21.15.49:0/300330118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x557005530d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- 172.21.15.49:0/300330118 >> 172.21.15.49:0/300330118 conn(0x557005522880 msgr2=0x557005521070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- 172.21.15.49:0/300330118 shutdown_connections 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- 172.21.15.49:0/300330118 wait complete. 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 -- start start 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 Processor -- start 2024-03-16T08:18:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x55700552be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.182+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x55700552a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ab9f0 con 0x557005531650 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ace60 con 0x557005528a70 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ac0c0 con 0x557005528150 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 10 client.0 _ll_register_callbacks cb 0x7f3b28009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40656/0 (socket says 172.21.15.49:40656) 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/139697471 learned_addr learned my addr 172.21.15.49:0/139697471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x55700552a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x55700552be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/139697471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 msgr2=0x55700552be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x55700552be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 msgr2=0x55700552a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x55700552a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 -- 172.21.15.49:0/139697471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570054539d0 con 0x557005531650 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b37fff640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x55700552a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x55700552be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b3ed9d640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b380bd000 tx=0x7f3b380b9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f098be40b585c912 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380bc9d0 con 0x557005531650 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3b380de360 con 0x557005531650 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.183+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b28010de0 con 0x557005531650 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3b3803a6f0 con 0x557005531650 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 10 client.16056 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5570055a6e20 con 0x557005531650 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b28010de0 con 0x557005531650 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3b380de040 con 0x557005531650 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 10 client.16056 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 20 client.16056 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5570055a7520 0x5570055a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 4 client.16056 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557005493d90 con 0x5570055a7520 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.184+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5570055a7520 0x5570055a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b3e59c640 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5570055a7520 0x5570055a9940 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d338edb9301dcba0 server_cookie=ca55a0d94e12e9c2 in_seq=0 out_seq=0 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b25ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b25ffb640 1 -- 172.21.15.49:0/139697471 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557005493d90 con 0x5570055a7520 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b25ffb640 10 client.16056 handle_command_reply: tid=0 2024-03-16T08:18:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 client.16056 shutdown 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 msgr2=0x55700552ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b380bd000 tx=0x7f3b380b9310 comp rx=0 tx=0).stop 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 shutdown_connections 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5570055a7520 0x5570055a9940 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557005531650 0x55700552ae20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557005528a70 0x55700552a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b25ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 --2- 172.21.15.49:0/139697471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557005528150 0x55700552be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 >> 172.21.15.49:0/139697471 conn(0x557005522880 msgr2=0x557005523c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 shutdown_connections 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 1 -- 172.21.15.49:0/139697471 wait complete. 2024-03-16T08:18:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.185+0000 7f3b479eb740 20 client.16056 trim_cache size 0 max 16384 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "41cc94ce-e7ed-43c9-878b-b1c15b3ef2ec": { 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "41cc94ce-e7ed-43c9-878b-b1c15b3ef2ec", 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:18:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 msgr2=0x7f3b2c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 0x7f3b2c0a4e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28013460 comp rx=0 tx=0).stop 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 msgr2=0x7f3b401a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b380348f0 tx=0x7f3b38032c90 comp rx=0 tx=0).stop 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 shutdown_connections 2024-03-16T08:18:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3b2c0a2970 0x7f3b2c0a4e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b40106660 0x7f3b401a3f70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b40101010 0x7f3b40199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 --2- 172.21.15.49:0/320770521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b40100650 0x7f3b40198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 >> 172.21.15.49:0/320770521 conn(0x7f3b400777e0 msgr2=0x7f3b40102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 shutdown_connections 2024-03-16T08:18:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:18:44.186+0000 7f3b44d22640 1 -- 172.21.15.49:0/320770521 wait complete. 2024-03-16T08:18:44.198 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:43 smithi049 ceph-mon[31942]: pgmap v48: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 176 KiB/s wr, 6 op/s 2024-03-16T08:18:44.199 INFO:tasks.cephfs.filesystem:scrub status for tag:41cc94ce-e7ed-43c9-878b-b1c15b3ef2ec - {'path': '/', 'tag': '41cc94ce-e7ed-43c9-878b-b1c15b3ef2ec', 'options': 'recursive,force'} 2024-03-16T08:18:45.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:44 smithi129 ceph-mon[36561]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-03-16T08:18:45.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:44 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:18:45.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:44 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:45.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:44 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:18:45.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:44 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:45.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:44 smithi154 ceph-mon[36471]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-03-16T08:18:45.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:44 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:18:45.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:44 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:45.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:44 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:18:45.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:44 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:44 smithi049 ceph-mon[31942]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-03-16T08:18:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:44 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:18:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:44 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:44 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:18:45.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:44 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:18:46.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:45 smithi129 ceph-mon[36561]: pgmap v49: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 608 KiB/s wr, 11 op/s 2024-03-16T08:18:46.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:45 smithi154 ceph-mon[36471]: pgmap v49: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 608 KiB/s wr, 11 op/s 2024-03-16T08:18:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:45 smithi049 ceph-mon[31942]: pgmap v49: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 608 KiB/s wr, 11 op/s 2024-03-16T08:18:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 -- 172.21.15.49:0/1320617254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc0100930 msgr2=0x7f8fc0100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 --2- 172.21.15.49:0/1320617254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc0100930 0x7f8fc0100db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009900 tx=0x7f8fbc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 -- 172.21.15.49:0/1320617254 shutdown_connections 2024-03-16T08:18:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 --2- 172.21.15.49:0/1320617254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc01012f0 0x7f8fc010d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 --2- 172.21.15.49:0/1320617254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc0100930 0x7f8fc0100db0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 --2- 172.21.15.49:0/1320617254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0105680 0x7f8fc0105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 -- 172.21.15.49:0/1320617254 >> 172.21.15.49:0/1320617254 conn(0x7f8fc0077530 msgr2=0x7f8fc0077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 -- 172.21.15.49:0/1320617254 shutdown_connections 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.085+0000 7f8fc747e640 1 -- 172.21.15.49:0/1320617254 wait complete. 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 Processor -- start 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 -- start start 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 0x7f8fc0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc0105680 0x7f8fc01a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0110ba0 con 0x7f8fc01012f0 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0110a20 con 0x7f8fc0100930 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc747e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0110d20 con 0x7f8fc0105680 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40672/0 (socket says 172.21.15.49:40672) 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc4a19640 1 -- 172.21.15.49:0/2515472370 learned_addr learned my addr 172.21.15.49:0/2515472370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 0x7f8fc0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.086+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc0105680 0x7f8fc01a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 -- 172.21.15.49:0/2515472370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc0105680 msgr2=0x7f8fc01a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc0105680 0x7f8fc01a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 -- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 msgr2=0x7f8fc0199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 -- 172.21.15.49:0/2515472370 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc01a4480 con 0x7f8fc0100930 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 0x7f8fc0198ec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9f0 tx=0x7f8fb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fb67fc640 1 -- 172.21.15.49:0/2515472370 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000ccc0 con 0x7f8fc0100930 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fb67fc640 1 -- 172.21.15.49:0/2515472370 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0010910 con 0x7f8fc0100930 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01a4650 con 0x7f8fc0100930 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fb67fc640 1 -- 172.21.15.49:0/2515472370 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018960 con 0x7f8fc0100930 2024-03-16T08:18:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.087+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01a4b10 con 0x7f8fc0100930 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.089+0000 7f8fb67fc640 1 -- 172.21.15.49:0/2515472370 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8fb0018b00 con 0x7f8fc0100930 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 Processor -- start 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 -- start start 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fb67fc640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 0x7f8fac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 0x55bf5a126c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a07fa40 con 0x55bf5a11b130 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a028920 con 0x55bf5a124820 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fca069740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a07bae0 con 0x55bf5a11ba50 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 0x7f8fac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fb67fc640 1 -- 172.21.15.49:0/2515472370 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8fb00cf0d0 con 0x7f8fc0100930 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 0x55bf5a126c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42766/0 (socket says 172.21.15.49:42766) 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc4a19640 1 -- 172.21.15.49:0/3087151965 learned_addr learned my addr 172.21.15.49:0/3087151965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 0x55bf5a126c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58370/0 (socket says 172.21.15.49:58370) 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40674/0 (socket says 172.21.15.49:40674) 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc4a19640 1 -- 172.21.15.49:0/3087151965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 msgr2=0x55bf5a126c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.090+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 0x55bf5a126c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc4a19640 1 -- 172.21.15.49:0/3087151965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 msgr2=0x55bf5a11b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc4a19640 1 -- 172.21.15.49:0/3087151965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5a040460 con 0x55bf5a11ba50 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc521a640 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 0x7f8fac0a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005fd0 tx=0x7f8fb8009e50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc036040 tx=0x7f8fbc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ad88967f0fc7f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fab7fe640 1 -- 172.21.15.49:0/3087151965 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc042070 con 0x55bf5a11ba50 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fab7fe640 1 -- 172.21.15.49:0/3087151965 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc004a70 con 0x55bf5a11ba50 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.091+0000 7f8fab7fe640 1 -- 172.21.15.49:0/3087151965 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc005160 con 0x55bf5a11ba50 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- 172.21.15.49:0/3087151965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 msgr2=0x55bf5a1242e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc036040 tx=0x7f8fbc009560 comp rx=0 tx=0).stop 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- 172.21.15.49:0/3087151965 shutdown_connections 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a124820 0x55bf5a126c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11ba50 0x55bf5a1242e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 --2- 172.21.15.49:0/3087151965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a11b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- 172.21.15.49:0/3087151965 >> 172.21.15.49:0/3087151965 conn(0x55bf5a115a50 msgr2=0x55bf5a115e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- 172.21.15.49:0/3087151965 shutdown_connections 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- 172.21.15.49:0/3087151965 wait complete. 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 -- start start 2024-03-16T08:18:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.092+0000 7f8fca069740 1 Processor -- start 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a1218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a1273f0 con 0x55bf5a124820 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a03bd70 con 0x55bf5a11ba50 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bf5a127570 con 0x55bf5a11b130 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fca069740 10 client.0 _ll_register_callbacks cb 0x55bf59d93a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58376/0 (socket says 172.21.15.49:58376) 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a1218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 -- 172.21.15.49:0/4106072436 learned_addr learned my addr 172.21.15.49:0/4106072436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40680/0 (socket says 172.21.15.49:40680) 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 -- 172.21.15.49:0/4106072436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 msgr2=0x55bf5a1218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a1218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 -- 172.21.15.49:0/4106072436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 msgr2=0x55bf5a122360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 -- 172.21.15.49:0/4106072436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf59d94210 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006010 tx=0x7f8fbc02cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119f79ebb37d144d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.093+0000 7f8fa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc042040 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8fbc0054a0 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc004a70 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc047460 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 10 client.25949 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf59e60620 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc004a70 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8fbc0596c0 con 0x55bf5a11ba50 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 10 client.25949 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 20 client.25949 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a134860 0x55bf5a136c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 4 client.25949 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5a08cf80 con 0x55bf5a134860 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.094+0000 7f8fc521a640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a134860 0x55bf5a136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.095+0000 7f8fa8ff9640 10 client.25949 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.095+0000 7f8fc521a640 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a134860 0x55bf5a136c50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e780ee75654ece11 server_cookie=623f1478adb7415a in_seq=0 out_seq=0 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.095+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/4106072436 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf5a08cf80 con 0x55bf5a134860 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.095+0000 7f8fa8ff9640 10 client.25949 handle_command_reply: tid=0 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 client.25949 shutdown 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 msgr2=0x55bf5a121e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006010 tx=0x7f8fbc02cc90 comp rx=0 tx=0).stop 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 shutdown_connections 2024-03-16T08:18:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a134860 0x55bf5a136c50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fa8ff9640 0 client.25949 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a121e20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 --2- 172.21.15.49:0/4106072436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a1218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 >> 172.21.15.49:0/4106072436 conn(0x55bf5a115a50 msgr2=0x55bf5a1177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 shutdown_connections 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 1 -- 172.21.15.49:0/4106072436 wait complete. 2024-03-16T08:18:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.096+0000 7f8fca069740 20 client.25949 trim_cache size 0 max 16384 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.111+0000 7f8fca069740 1 Processor -- start 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.111+0000 7f8fca069740 1 -- start start 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a130190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 0x55bf5a13c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01afb10 con 0x55bf5a11b130 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01ad3e0 con 0x55bf5a11ba50 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fca069740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01ab210 con 0x55bf5a124820 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a130190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a130190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40694/0 (socket says 172.21.15.49:40694) 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 0x55bf5a13c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc521a640 1 -- 172.21.15.49:0/1440290859 learned_addr learned my addr 172.21.15.49:0/1440290859 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58378/0 (socket says 172.21.15.49:58378) 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 -- 172.21.15.49:0/1440290859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 msgr2=0x55bf5a13c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 0x55bf5a13c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 -- 172.21.15.49:0/1440290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 msgr2=0x55bf5a130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a130190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 -- 172.21.15.49:0/1440290859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5a115580 con 0x55bf5a11ba50 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 0x55bf5a13c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.112+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005ec0 tx=0x7f8fbc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce0ca47607d6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8faaffd640 1 -- 172.21.15.49:0/1440290859 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004a70 con 0x55bf5a11ba50 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8faaffd640 1 -- 172.21.15.49:0/1440290859 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc05e020 con 0x55bf5a11ba50 2024-03-16T08:18:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8faaffd640 1 -- 172.21.15.49:0/1440290859 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc047600 con 0x55bf5a11ba50 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 -- 172.21.15.49:0/1440290859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 msgr2=0x55bf5a1175b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005ec0 tx=0x7f8fbc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 -- 172.21.15.49:0/1440290859 shutdown_connections 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a124820 0x55bf5a13c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a1175b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 --2- 172.21.15.49:0/1440290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a11b130 0x55bf5a130190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 -- 172.21.15.49:0/1440290859 >> 172.21.15.49:0/1440290859 conn(0x55bf5a115a50 msgr2=0x55bf5a116ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 -- 172.21.15.49:0/1440290859 shutdown_connections 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.113+0000 7f8fca069740 1 -- 172.21.15.49:0/1440290859 wait complete. 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 -- start start 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 Processor -- start 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a128200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a128c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01ab8a0 con 0x55bf5a124820 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01acd10 con 0x55bf5a11ba50 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01abf70 con 0x55bf5a11b130 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fca069740 10 client.0 _ll_register_callbacks cb 0x7f8fb00973b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fc5a1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a128c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fc4a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58388/0 (socket says 172.21.15.49:58388) 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc4a19640 1 -- 172.21.15.49:0/2048475353 learned_addr learned my addr 172.21.15.49:0/2048475353 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.114+0000 7f8fc521a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a128200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 -- 172.21.15.49:0/2048475353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 msgr2=0x55bf5a128200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a128200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 -- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 msgr2=0x55bf5a128740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:18:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 -- 172.21.15.49:0/2048475353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5a1999e0 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a128200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc4a19640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fc5a1b640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a128c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009270 tx=0x7f8fb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb32a57a07541079 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8003970 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8fb801c040 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc05e020 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fb8003b10 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fca069740 10 client.16074 fetch_fsmap learned FSMap version 24 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.115+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf5a19acd0 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc05e020 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8fb8029070 con 0x55bf5a124820 2024-03-16T08:18:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 10 client.16074 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 20 client.16074 populate_metadata read hostname 'smithi049' 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a19b4d0 0x55bf5a19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 4 client.16074 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf5a0872d0 con 0x55bf5a19b4d0 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a19b4d0 0x55bf5a19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fa8ff9640 10 client.16074 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.116+0000 7f8fc521a640 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a19b4d0 0x55bf5a19d8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e480da717d0af5d5 server_cookie=b7644a74d51a2e20 in_seq=0 out_seq=0 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.125+0000 7f8fa8ff9640 1 -- 172.21.15.49:0/2048475353 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf5a0872d0 con 0x55bf5a19b4d0 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.125+0000 7f8fa8ff9640 10 client.16074 handle_command_reply: tid=0 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.125+0000 7f8fca069740 1 client.16074 shutdown 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.125+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 msgr2=0x55bf5a128c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.125+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a128c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009270 tx=0x7f8fb8004ab0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 shutdown_connections 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55bf5a19b4d0 0x55bf5a19d8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bf5a124820 0x55bf5a128c80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bf5a11ba50 0x55bf5a128740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fa8ff9640 0 client.16074 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 --2- 172.21.15.49:0/2048475353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bf5a11b130 0x55bf5a128200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 >> 172.21.15.49:0/2048475353 conn(0x55bf5a115a50 msgr2=0x55bf5a114bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 shutdown_connections 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 1 -- 172.21.15.49:0/2048475353 wait complete. 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fca069740 20 client.16074 trim_cache size 0 max 16384 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:18:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:18:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 msgr2=0x7f8fac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 0x7f8fac0a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005fd0 tx=0x7f8fb8009e50 comp rx=0 tx=0).stop 2024-03-16T08:18:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 msgr2=0x7f8fc0198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.126+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 0x7f8fc0198ec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9f0 tx=0x7f8fb000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 shutdown_connections 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fac0a2a10 0x7f8fac0a4ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc0105680 0x7f8fc01a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc01012f0 0x7f8fc0199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 --2- 172.21.15.49:0/2515472370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fc0100930 0x7f8fc0198ec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 >> 172.21.15.49:0/2515472370 conn(0x7f8fc0077530 msgr2=0x7f8fc00fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 shutdown_connections 2024-03-16T08:18:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:18:47.127+0000 7f8fc747e640 1 -- 172.21.15.49:0/2515472370 wait complete. 2024-03-16T08:18:48.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:47 smithi129 ceph-mon[36561]: pgmap v50: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 608 KiB/s wr, 9 op/s 2024-03-16T08:18:48.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:47 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:18:48.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:47 smithi154 ceph-mon[36471]: pgmap v50: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 608 KiB/s wr, 9 op/s 2024-03-16T08:18:48.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:47 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:18:48.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:47 smithi049 ceph-mon[31942]: pgmap v50: 97 pgs: 97 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 608 KiB/s wr, 9 op/s 2024-03-16T08:18:48.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:47 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:18:50.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:49 smithi154 ceph-mon[36471]: pgmap v51: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.0 MiB/s wr, 17 op/s 2024-03-16T08:18:50.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:49 smithi049 ceph-mon[31942]: pgmap v51: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.0 MiB/s wr, 17 op/s 2024-03-16T08:18:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:49 smithi129 ceph-mon[36561]: pgmap v51: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.0 MiB/s wr, 17 op/s 2024-03-16T08:18:52.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:51 smithi154 ceph-mon[36471]: pgmap v52: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:52.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:51 smithi049 ceph-mon[31942]: pgmap v52: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:52.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:51 smithi129 ceph-mon[36561]: pgmap v52: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:53 smithi154 ceph-mon[36471]: pgmap v53: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:53 smithi049 ceph-mon[31942]: pgmap v53: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:53 smithi129 ceph-mon[36561]: pgmap v53: 97 pgs: 97 active+clean; 2.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-16T08:18:56.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:55 smithi154 ceph-mon[36471]: pgmap v54: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-03-16T08:18:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:55 smithi049 ceph-mon[31942]: pgmap v54: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-03-16T08:18:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:55 smithi129 ceph-mon[36561]: pgmap v54: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-03-16T08:18:58.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:57 smithi154 ceph-mon[36471]: pgmap v55: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-16T08:18:58.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:57 smithi049 ceph-mon[31942]: pgmap v55: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-16T08:18:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:57 smithi129 ceph-mon[36561]: pgmap v55: 97 pgs: 97 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-16T08:19:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 -- 172.21.15.49:0/1645672695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 msgr2=0x7f336410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 --2- 172.21.15.49:0/1645672695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f336410e230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f335c009ca0 tx=0x7f335c02c230 comp rx=0 tx=0).stop 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 -- 172.21.15.49:0/1645672695 shutdown_connections 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 --2- 172.21.15.49:0/1645672695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f336410e230 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 --2- 172.21.15.49:0/1645672695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364103370 0x7f336410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 --2- 172.21.15.49:0/1645672695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364102a50 0x7f3364102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.220+0000 7f336b9e6640 1 -- 172.21.15.49:0/1645672695 >> 172.21.15.49:0/1645672695 conn(0x7f33640777c0 msgr2=0x7f3364077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- 172.21.15.49:0/1645672695 shutdown_connections 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- 172.21.15.49:0/1645672695 wait complete. 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 Processor -- start 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- start start 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364102a50 0x7f33641006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364103370 0x7f3364100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3364111b90 con 0x7f336410bd90 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3364111a10 con 0x7f3364102a50 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.221+0000 7f336b9e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3364111d10 con 0x7f3364103370 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3368f81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364103370 0x7f3364100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59512/0 (socket says 172.21.15.49:59512) 2024-03-16T08:19:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369782640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364102a50 0x7f33641006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 -- 172.21.15.49:0/3530047350 learned_addr learned my addr 172.21.15.49:0/3530047350 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 -- 172.21.15.49:0/3530047350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364103370 msgr2=0x7f3364100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364103370 0x7f3364100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 -- 172.21.15.49:0/3530047350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364102a50 msgr2=0x7f33641006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364102a50 0x7f33641006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 -- 172.21.15.49:0/3530047350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33641a6410 con 0x7f336410bd90 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f3369f83640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f335c02c840 tx=0x7f335c035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f335a7fc640 1 -- 172.21.15.49:0/3530047350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c039040 con 0x7f336410bd90 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641a66a0 con 0x7f336410bd90 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f335a7fc640 1 -- 172.21.15.49:0/3530047350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c044070 con 0x7f336410bd90 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f335a7fc640 1 -- 172.21.15.49:0/3530047350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0042a0 con 0x7f336410bd90 2024-03-16T08:19:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.222+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a6b90 con 0x7f336410bd90 2024-03-16T08:19:00.227 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:18:59 smithi154 ceph-mon[36471]: pgmap v56: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f335a7fc640 1 -- 172.21.15.49:0/3530047350 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f335c004440 con 0x7f336410bd90 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 Processor -- start 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 -- start start 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bdac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556313b33a40 con 0x556313bcfa50 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556313adc920 con 0x556313bd8820 2024-03-16T08:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f336e5cf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556313b2fae0 con 0x556313bcf130 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.225+0000 7f335a7fc640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 0x7f33500a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59522/0 (socket says 172.21.15.49:59522) 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47088/0 (socket says 172.21.15.49:47088) 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369782640 1 -- 172.21.15.49:0/1163713677 learned_addr learned my addr 172.21.15.49:0/1163713677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bdac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369f83640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 0x7f33500a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f335a7fc640 1 -- 172.21.15.49:0/3530047350 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f335c0f10f0 con 0x7f336410bd90 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 -- 172.21.15.49:0/1163713677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 msgr2=0x556313bdac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bdac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 -- 172.21.15.49:0/1163713677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 msgr2=0x556313bd82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 -- 172.21.15.49:0/1163713677 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556313af4460 con 0x556313bcf130 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369782640 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3368f81640 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33641020e0 tx=0x7f3360004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624cee29c81dc192 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f3369f83640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 0x7f33500a4f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f335c0bd840 tx=0x7f335c02c230 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f334f7fe640 1 -- 172.21.15.49:0/1163713677 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360005020 con 0x556313bcf130 2024-03-16T08:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f334f7fe640 1 -- 172.21.15.49:0/1163713677 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33600047a0 con 0x556313bcf130 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.226+0000 7f334f7fe640 1 -- 172.21.15.49:0/1163713677 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000f6e0 con 0x556313bcf130 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 -- 172.21.15.49:0/1163713677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 msgr2=0x556313bcf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33641020e0 tx=0x7f3360004730 comp rx=0 tx=0).stop 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 -- 172.21.15.49:0/1163713677 shutdown_connections 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bdac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 --2- 172.21.15.49:0/1163713677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bcf510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 -- 172.21.15.49:0/1163713677 >> 172.21.15.49:0/1163713677 conn(0x556313bc9a50 msgr2=0x556313bc9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 -- 172.21.15.49:0/1163713677 shutdown_connections 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.227+0000 7f336e5cf740 1 -- 172.21.15.49:0/1163713677 wait complete. 2024-03-16T08:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 -- start start 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 Processor -- start 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcf130 0x556313bd58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bd8820 0x556313bd6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556313bdb450 con 0x556313bcfa50 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556313aefd70 con 0x556313bcf130 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556313bdb5d0 con 0x556313bd8820 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f336e5cf740 10 client.0 _ll_register_callbacks cb 0x556313847a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3369782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3369782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59538/0 (socket says 172.21.15.49:59538) 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3369782640 1 -- 172.21.15.49:0/2880078762 learned_addr learned my addr 172.21.15.49:0/2880078762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3368f81640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bd8820 0x556313bd6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3368f81640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcf130 0x556313bd58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3369782640 1 -- 172.21.15.49:0/2880078762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bd8820 msgr2=0x556313bd6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.228+0000 7f3369782640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bd8820 0x556313bd6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f3369782640 1 -- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcf130 msgr2=0x556313bd58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f3369782640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcf130 0x556313bd58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f3369782640 1 -- 172.21.15.49:0/2880078762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556313848210 con 0x556313bcfa50 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f3369782640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3354012840 tx=0x7f3354012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeebb4e84df1870 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400f920 con 0x556313bcfa50 2024-03-16T08:19:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f335400fac0 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33600047a0 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f335400fcc0 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f336e5cf740 10 client.16086 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556313914620 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33600047a0 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f335401c420 con 0x556313bcfa50 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.229+0000 7f336e5cf740 10 client.16086 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f336e5cf740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f336e5cf740 20 client.16086 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313be8a30 0x556313beae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f336e5cf740 4 client.16086 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556313b40f80 con 0x556313be8a30 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f3368f81640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313be8a30 0x556313beae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f334cff9640 10 client.16086 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.230+0000 7f3368f81640 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313be8a30 0x556313beae80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7866dbde70402d90 server_cookie=3d1dcd736ac03fe0 in_seq=0 out_seq=0 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.231+0000 7f334cff9640 1 -- 172.21.15.49:0/2880078762 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556313b40f80 con 0x556313be8a30 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.231+0000 7f334cff9640 10 client.16086 handle_command_reply: tid=0 2024-03-16T08:19:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.231+0000 7f336e5cf740 1 client.16086 shutdown 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.231+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 msgr2=0x556313bd5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.231+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3354012840 tx=0x7f3354012d00 comp rx=0 tx=0).stop 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 shutdown_connections 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313be8a30 0x556313beae80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bd8820 0x556313bd6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcfa50 0x556313bd5df0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f334cff9640 0 client.16086 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 --2- 172.21.15.49:0/2880078762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcf130 0x556313bd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 >> 172.21.15.49:0/2880078762 conn(0x556313bc9a50 msgr2=0x556313bcb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 shutdown_connections 2024-03-16T08:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 1 -- 172.21.15.49:0/2880078762 wait complete. 2024-03-16T08:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.232+0000 7f336e5cf740 20 client.16086 trim_cache size 0 max 16384 2024-03-16T08:19:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 Processor -- start 2024-03-16T08:19:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 -- start start 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bf0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33641b1da0 con 0x556313bcf130 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33641af670 con 0x556313bd8820 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f336e5cf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ad4a0 con 0x556313bcfa50 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f3369782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f3368f81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f3369782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:47104/0 (socket says 172.21.15.49:47104) 2024-03-16T08:19:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f3369782640 1 -- 172.21.15.49:0/852275660 learned_addr learned my addr 172.21.15.49:0/852275660 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.247+0000 7f3368f81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59552/0 (socket says 172.21.15.49:59552) 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3368f81640 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bf0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 -- 172.21.15.49:0/852275660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 msgr2=0x556313bf0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bf0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 -- 172.21.15.49:0/852275660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 msgr2=0x556313be41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 -- 172.21.15.49:0/852275660 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556313beb500 con 0x556313bcfa50 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3368f81640 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f3369782640 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c688f3de7864b5f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f334effd640 1 -- 172.21.15.49:0/852275660 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354029020 con 0x556313bcfa50 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f334effd640 1 -- 172.21.15.49:0/852275660 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33540123a0 con 0x556313bcfa50 2024-03-16T08:19:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.248+0000 7f334effd640 1 -- 172.21.15.49:0/852275660 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335401c420 con 0x556313bcfa50 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- 172.21.15.49:0/852275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 msgr2=0x556313bd76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354009f40 comp rx=0 tx=0).stop 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- 172.21.15.49:0/852275660 shutdown_connections 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bd8820 0x556313bf0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcfa50 0x556313bd76e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 --2- 172.21.15.49:0/852275660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bcf130 0x556313be41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- 172.21.15.49:0/852275660 >> 172.21.15.49:0/852275660 conn(0x556313bc9a50 msgr2=0x556313bd6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- 172.21.15.49:0/852275660 shutdown_connections 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- 172.21.15.49:0/852275660 wait complete. 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 -- start start 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.249+0000 7f336e5cf740 1 Processor -- start 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bdc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 0x556313bdc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33641adb30 con 0x556313bd8820 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33641aefa0 con 0x556313bcfa50 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ae200 con 0x556313bcf130 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f336e5cf740 10 client.0 _ll_register_callbacks cb 0x7f3354009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59558/0 (socket says 172.21.15.49:59558) 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 -- 172.21.15.49:0/3973736824 learned_addr learned my addr 172.21.15.49:0/3973736824 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3369782640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 0x556313bdc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bdc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 -- 172.21.15.49:0/3973736824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 msgr2=0x556313bdc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bdc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 -- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 msgr2=0x556313bdc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 0x556313bdc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3368f81640 1 -- 172.21.15.49:0/3973736824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556313bdd2f0 con 0x556313bd8820 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.250+0000 7f3369782640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 0x556313bdc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f3368f81640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3360011fd0 tx=0x7f3360009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6b652dd804486 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336002e490 con 0x556313bd8820 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3360027040 con 0x556313bd8820 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33540123a0 con 0x556313bd8820 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f336002e630 con 0x556313bd8820 2024-03-16T08:19:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 10 client.16092 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556313c4f410 con 0x556313bd8820 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33540123a0 con 0x556313bd8820 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f336001d030 con 0x556313bd8820 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 10 client.16092 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 20 client.16092 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.251+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313c4f8e0 0x556313c51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.252+0000 7f336e5cf740 4 client.16092 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.252+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556313b3b2d0 con 0x556313c4f8e0 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.252+0000 7f3369782640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313c4f8e0 0x556313c51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.252+0000 7f334cff9640 10 client.16092 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.252+0000 7f3369782640 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313c4f8e0 0x556313c51cd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabdbf9573e59e0b server_cookie=50c2faf91ecb506a in_seq=0 out_seq=0 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f334cff9640 1 -- 172.21.15.49:0/3973736824 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556313b3b2d0 con 0x556313c4f8e0 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f334cff9640 10 client.16092 handle_command_reply: tid=0 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 client.16092 shutdown 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 msgr2=0x556313c4ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3360011fd0 tx=0x7f3360009b30 comp rx=0 tx=0).stop 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 shutdown_connections 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556313c4f8e0 0x556313c51cd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556313bd8820 0x556313c4ddc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556313bcfa50 0x556313bdc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f334cff9640 0 client.16092 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 --2- 172.21.15.49:0/3973736824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556313bcf130 0x556313bdc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 >> 172.21.15.49:0/3973736824 conn(0x556313bc9a50 msgr2=0x556313bc8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 shutdown_connections 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.262+0000 7f336e5cf740 1 -- 172.21.15.49:0/3973736824 wait complete. 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336e5cf740 20 client.16092 trim_cache size 0 max 16384 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:19:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 msgr2=0x7f33500a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 0x7f33500a4f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f335c0bd840 tx=0x7f335c02c230 comp rx=0 tx=0).stop 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 msgr2=0x7f3364101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f335c02c840 tx=0x7f335c035d90 comp rx=0 tx=0).stop 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 shutdown_connections 2024-03-16T08:19:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f33500a2a40 0x7f33500a4f00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336410bd90 0x7f3364101120 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3364103370 0x7f3364100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 --2- 172.21.15.49:0/3530047350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3364102a50 0x7f33641006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 >> 172.21.15.49:0/3530047350 conn(0x7f33640777c0 msgr2=0x7f33640fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 shutdown_connections 2024-03-16T08:19:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:00.263+0000 7f336b9e6640 1 -- 172.21.15.49:0/3530047350 wait complete. 2024-03-16T08:19:00.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:18:59 smithi049 ceph-mon[31942]: pgmap v56: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-03-16T08:19:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:18:59 smithi129 ceph-mon[36561]: pgmap v56: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-03-16T08:19:02.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:01 smithi154 ceph-mon[36471]: pgmap v57: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:02.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:01 smithi049 ceph-mon[31942]: pgmap v57: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:01 smithi129 ceph-mon[36561]: pgmap v57: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:04.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:03 smithi154 ceph-mon[36471]: pgmap v58: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:03 smithi049 ceph-mon[31942]: pgmap v58: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:03 smithi129 ceph-mon[36561]: pgmap v58: 97 pgs: 97 active+clean; 179 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-03-16T08:19:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:05 smithi154 ceph-mon[36471]: pgmap v59: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 277 op/s 2024-03-16T08:19:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:05 smithi049 ceph-mon[31942]: pgmap v59: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 277 op/s 2024-03-16T08:19:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:05 smithi129 ceph-mon[36561]: pgmap v59: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 277 op/s 2024-03-16T08:19:08.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:07 smithi154 ceph-mon[36471]: pgmap v60: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 273 op/s 2024-03-16T08:19:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:07 smithi049 ceph-mon[31942]: pgmap v60: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 273 op/s 2024-03-16T08:19:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:07 smithi129 ceph-mon[36561]: pgmap v60: 97 pgs: 97 active+clean; 258 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 273 op/s 2024-03-16T08:19:10.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:09 smithi154 ceph-mon[36471]: pgmap v61: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 609 op/s 2024-03-16T08:19:10.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:09 smithi049 ceph-mon[31942]: pgmap v61: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 609 op/s 2024-03-16T08:19:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:09 smithi129 ceph-mon[36561]: pgmap v61: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 609 op/s 2024-03-16T08:19:12.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:11 smithi154 ceph-mon[36471]: pgmap v62: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:12.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:11 smithi049 ceph-mon[31942]: pgmap v62: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:11 smithi129 ceph-mon[36561]: pgmap v62: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.361+0000 7fa85ccdc640 1 -- 172.21.15.49:0/2581708054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 msgr2=0x7fa858100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.361+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/2581708054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858100db0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa84002bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 -- 172.21.15.49:0/2581708054 shutdown_connections 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/2581708054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8581012f0 0x7fa85810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/2581708054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858100db0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/2581708054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa858105680 0x7fa858105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 -- 172.21.15.49:0/2581708054 >> 172.21.15.49:0/2581708054 conn(0x7fa858077530 msgr2=0x7fa858077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 -- 172.21.15.49:0/2581708054 shutdown_connections 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 -- 172.21.15.49:0/2581708054 wait complete. 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 Processor -- start 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.362+0000 7fa85ccdc640 1 -- start start 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581012f0 0x7fa858199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa858105680 0x7fa8581a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa858110b60 con 0x7fa858100930 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581109e0 con 0x7fa858105680 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85ccdc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa858110ce0 con 0x7fa8581012f0 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581012f0 0x7fa858199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55108/0 (socket says 172.21.15.49:55108) 2024-03-16T08:19:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 -- 172.21.15.49:0/3579737101 learned_addr learned my addr 172.21.15.49:0/3579737101 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa856d9d640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa858105680 0x7fa8581a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 -- 172.21.15.49:0/3579737101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581012f0 msgr2=0x7fa858199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581012f0 0x7fa858199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 -- 172.21.15.49:0/3579737101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa858105680 msgr2=0x7fa8581a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa858105680 0x7fa8581a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 -- 172.21.15.49:0/3579737101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a4480 con 0x7fa858100930 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.363+0000 7fa85659c640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.364+0000 7fa84f7fe640 1 -- 172.21.15.49:0/3579737101 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800cd80 con 0x7fa858100930 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.364+0000 7fa84f7fe640 1 -- 172.21.15.49:0/3579737101 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa848010910 con 0x7fa858100930 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.364+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a4650 con 0x7fa858100930 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.364+0000 7fa84f7fe640 1 -- 172.21.15.49:0/3579737101 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848018990 con 0x7fa858100930 2024-03-16T08:19:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.364+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a4b10 con 0x7fa858100930 2024-03-16T08:19:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.366+0000 7fa84f7fe640 1 -- 172.21.15.49:0/3579737101 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa848010ac0 con 0x7fa858100930 2024-03-16T08:19:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.366+0000 7fa85f9a0740 1 Processor -- start 2024-03-16T08:19:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 -- start start 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56107c035a40 con 0x56107c0d1570 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56107bfde920 con 0x56107c0dec30 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56107c031ae0 con 0x56107c0d0c50 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa84f7fe640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 0x7fa8440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37500/0 (socket says 172.21.15.49:37500) 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55114/0 (socket says 172.21.15.49:55114) 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 -- 172.21.15.49:0/1955336989 learned_addr learned my addr 172.21.15.49:0/1955336989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa84ffff640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 0x7fa8440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa84f7fe640 1 -- 172.21.15.49:0/3579737101 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa8480cf030 con 0x7fa858100930 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 -- 172.21.15.49:0/1955336989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 msgr2=0x56107c0e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 -- 172.21.15.49:0/1955336989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 msgr2=0x56107c0de6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa856d9d640 1 -- 172.21.15.49:0/1955336989 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107bff6460 con 0x56107c0d0c50 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.367+0000 7fa85659c640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa84ffff640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa856d9d640 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa850013b00 tx=0x7fa85000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dbfc63ebbe3f891 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa84cff9640 1 -- 172.21.15.49:0/1955336989 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850013540 con 0x56107c0d0c50 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa84cff9640 1 -- 172.21.15.49:0/1955336989 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850004400 con 0x56107c0d0c50 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa84cff9640 1 -- 172.21.15.49:0/1955336989 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850005410 con 0x56107c0d0c50 2024-03-16T08:19:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.368+0000 7fa856d9d640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 0x7fa8440a4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa850023000 tx=0x7fa850020030 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- 172.21.15.49:0/1955336989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 msgr2=0x56107c0d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa850013b00 tx=0x7fa85000f3c0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- 172.21.15.49:0/1955336989 shutdown_connections 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/1955336989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0d1030 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- 172.21.15.49:0/1955336989 >> 172.21.15.49:0/1955336989 conn(0x56107c0cba50 msgr2=0x56107c0cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- 172.21.15.49:0/1955336989 shutdown_connections 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- 172.21.15.49:0/1955336989 wait complete. 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.369+0000 7fa85f9a0740 1 -- start start 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 Processor -- start 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d1570 0x56107c0dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 0x56107c0dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56107c0e17f0 con 0x56107c0d0c50 2024-03-16T08:19:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56107bff1d70 con 0x56107c0d1570 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56107c0e1970 con 0x56107c0dec30 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85f9a0740 10 client.0 _ll_register_callbacks cb 0x56107bd49a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55128/0 (socket says 172.21.15.49:55128) 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 -- 172.21.15.49:0/2136180650 learned_addr learned my addr 172.21.15.49:0/2136180650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa856d9d640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 0x56107c0dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d1570 0x56107c0dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 -- 172.21.15.49:0/2136180650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 msgr2=0x56107c0dc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 0x56107c0dc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 -- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d1570 msgr2=0x56107c0dc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d1570 0x56107c0dc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa85659c640 1 -- 172.21.15.49:0/2136180650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107bd4a210 con 0x56107c0d0c50 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.370+0000 7fa856d9d640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 0x56107c0dc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85659c640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa8480cc830 tx=0x7fa848009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a64d8a25c0a26c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a3e80 con 0x56107c0d0c50 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa8480d10f0 con 0x56107c0d0c50 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa850004400 con 0x56107c0d0c50 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa848084cd0 con 0x56107c0d0c50 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 10 client.16104 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56107be16620 con 0x56107c0d0c50 2024-03-16T08:19:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850004400 con 0x56107c0d0c50 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa848018b30 con 0x56107c0d0c50 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 10 client.16104 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 20 client.16104 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c0f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 4 client.16104 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.371+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56107c042f80 con 0x56107c0eed90 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.372+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c0f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.372+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c0f1180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a6685fc785b3e0 server_cookie=857ed433fd424782 in_seq=0 out_seq=0 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.372+0000 7fa83dffb640 10 client.16104 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa83dffb640 1 -- 172.21.15.49:0/2136180650 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56107c042f80 con 0x56107c0eed90 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa83dffb640 10 client.16104 handle_command_reply: tid=0 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 client.16104 shutdown 2024-03-16T08:19:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 msgr2=0x56107c0dbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa8480cc830 tx=0x7fa848009310 comp rx=0 tx=0).stop 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 shutdown_connections 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c0f1180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa83dffb640 0 client.16104 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0dec30 0x56107c0dc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d1570 0x56107c0dc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2136180650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d0c50 0x56107c0dbc60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 >> 172.21.15.49:0/2136180650 conn(0x56107c0cba50 msgr2=0x56107c0cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 shutdown_connections 2024-03-16T08:19:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2136180650 wait complete. 2024-03-16T08:19:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.373+0000 7fa85f9a0740 20 client.16104 trim_cache size 0 max 16384 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 Processor -- start 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 -- start start 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 0x56107c0de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d1570 0x56107c0ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581adad0 con 0x56107c0dec30 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581ab3a0 con 0x56107c0d0c50 2024-03-16T08:19:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581a91d0 con 0x56107c0d1570 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa85659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 0x56107c0de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55140/0 (socket says 172.21.15.49:55140) 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa856d9d640 1 -- 172.21.15.49:0/92834735 learned_addr learned my addr 172.21.15.49:0/92834735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.393+0000 7fa84ffff640 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d1570 0x56107c0ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 -- 172.21.15.49:0/92834735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d1570 msgr2=0x56107c0ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d1570 0x56107c0ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 -- 172.21.15.49:0/92834735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 msgr2=0x56107c0de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 0x56107c0de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 -- 172.21.15.49:0/92834735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107c0d0410 con 0x56107c0dec30 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa85659c640 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 0x56107c0de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa856d9d640 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa850014fd0 tx=0x7fa850009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1df26691b2b98fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa83ffff640 1 -- 172.21.15.49:0/92834735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500130c0 con 0x56107c0dec30 2024-03-16T08:19:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa83ffff640 1 -- 172.21.15.49:0/92834735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850013260 con 0x56107c0dec30 2024-03-16T08:19:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.394+0000 7fa83ffff640 1 -- 172.21.15.49:0/92834735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850010690 con 0x56107c0dec30 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- 172.21.15.49:0/92834735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 msgr2=0x56107c0f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa850014fd0 tx=0x7fa850009b20 comp rx=0 tx=0).stop 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- 172.21.15.49:0/92834735 shutdown_connections 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0dec30 0x56107c0f6ac0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d1570 0x56107c0ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/92834735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0d0c50 0x56107c0de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- 172.21.15.49:0/92834735 >> 172.21.15.49:0/92834735 conn(0x56107c0cba50 msgr2=0x56107c0c6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- 172.21.15.49:0/92834735 shutdown_connections 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- 172.21.15.49:0/92834735 wait complete. 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 -- start start 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.395+0000 7fa85f9a0740 1 Processor -- start 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581a9860 con 0x56107c0d1570 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581aacd0 con 0x56107c0dec30 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581a9f30 con 0x56107c0d0c50 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85f9a0740 10 client.0 _ll_register_callbacks cb 0x7fa848009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37516/0 (socket says 172.21.15.49:37516) 2024-03-16T08:19:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85659c640 1 -- 172.21.15.49:0/2373479237 learned_addr learned my addr 172.21.15.49:0/2373479237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa856d9d640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 -- 172.21.15.49:0/2373479237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 msgr2=0x56107c0e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 -- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 msgr2=0x56107c0e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 -- 172.21.15.49:0/2373479237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107c0e29e0 con 0x56107c0d1570 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa85659c640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa84ffff640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0e6fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa84002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50fb75941620cd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.396+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84004f070 con 0x56107c0d1570 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa840004240 con 0x56107c0d1570 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa850013260 con 0x56107c0d1570 2024-03-16T08:19:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa84005b050 con 0x56107c0d1570 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 10 client.16110 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56107c154da0 con 0x56107c0d1570 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850013260 con 0x56107c0d1570 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa8400046c0 con 0x56107c0d1570 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 10 client.16110 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 20 client.16110 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 4 client.16110 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:19:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56107c03d2d0 con 0x56107c0eed90 2024-03-16T08:19:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.397+0000 7fa85659c640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.398+0000 7fa83dffb640 10 client.16110 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.398+0000 7fa85659c640 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c1574e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2169487c88bef4d server_cookie=47a25bcbc9e8fb66 in_seq=0 out_seq=0 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa83dffb640 1 -- 172.21.15.49:0/2373479237 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56107c03d2d0 con 0x56107c0eed90 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa83dffb640 10 client.16110 handle_command_reply: tid=0 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 client.16110 shutdown 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 msgr2=0x56107c0e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0e6fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa84002bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 shutdown_connections 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56107c0eed90 0x56107c1574e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56107c0dec30 0x56107c0e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa83dffb640 0 client.16110 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56107c0d1570 0x56107c0e6fe0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 --2- 172.21.15.49:0/2373479237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56107c0d0c50 0x56107c0e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 >> 172.21.15.49:0/2373479237 conn(0x56107c0cba50 msgr2=0x56107c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 shutdown_connections 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 1 -- 172.21.15.49:0/2373479237 wait complete. 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.411+0000 7fa85f9a0740 20 client.16110 trim_cache size 0 max 16384 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:19:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:19:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:19:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:19:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 msgr2=0x7fa8440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 0x7fa8440a4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa850023000 tx=0x7fa850020030 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 msgr2=0x7fa858198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 shutdown_connections 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa8440a2a10 0x7fa8440a4ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa858105680 0x7fa8581a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581012f0 0x7fa858199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 --2- 172.21.15.49:0/3579737101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa858100930 0x7fa858198ec0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 >> 172.21.15.49:0/3579737101 conn(0x7fa858077530 msgr2=0x7fa8580fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 shutdown_connections 2024-03-16T08:19:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:13.412+0000 7fa85ccdc640 1 -- 172.21.15.49:0/3579737101 wait complete. 2024-03-16T08:19:14.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:19:14.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:19:14.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:13 smithi154 ceph-mon[36471]: pgmap v63: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:14.230 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:13 smithi049 ceph-mon[31942]: pgmap v63: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:14.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2583432058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a80689c0 msgr2=0x7fa1a8068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2583432058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a80689c0 0x7fa1a8068e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa194009930 tx=0x7fa19402bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:14.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2583432058 shutdown_connections 2024-03-16T08:19:14.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2583432058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2583432058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a80689c0 0x7fa1a8068e40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2583432058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a8107760 0x7fa1a8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2583432058 >> 172.21.15.49:0/2583432058 conn(0x7fa1a80777e0 msgr2=0x7fa1a8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2583432058 shutdown_connections 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.323+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2583432058 wait complete. 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 Processor -- start 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 -- start start 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a80689c0 0x7fa1a81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 0x7fa1a819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8111b70 con 0x7fa1a8069380 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81119f0 con 0x7fa1a80689c0 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1aeec6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8111cf0 con 0x7fa1a8107760 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55174/0 (socket says 172.21.15.49:55174) 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1a7fff640 1 -- 172.21.15.49:0/2095070576 learned_addr learned my addr 172.21.15.49:0/2095070576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 0x7fa1a819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.324+0000 7fa1acc62640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a80689c0 0x7fa1a81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 -- 172.21.15.49:0/2095070576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a80689c0 msgr2=0x7fa1a81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a80689c0 0x7fa1a81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 -- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 msgr2=0x7fa1a81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 -- 172.21.15.49:0/2095070576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819d0a0 con 0x7fa1a8107760 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 0x7fa1a819c970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009fd0 tx=0x7fa1a000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1a5ffb640 1 -- 172.21.15.49:0/2095070576 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000ce20 con 0x7fa1a8107760 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1a5ffb640 1 -- 172.21.15.49:0/2095070576 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x7fa1a8107760 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819d390 con 0x7fa1a8107760 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1a5ffb640 1 -- 172.21.15.49:0/2095070576 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0010720 con 0x7fa1a8107760 2024-03-16T08:19:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.325+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a4c20 con 0x7fa1a8107760 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.327+0000 7fa1a5ffb640 1 -- 172.21.15.49:0/2095070576 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa1a0005540 con 0x7fa1a8107760 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 Processor -- start 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 -- start start 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1a5ffb640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 0x7fa1980a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1a5ffb640 1 -- 172.21.15.49:0/2095070576 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa1a00cf500 con 0x7fa1a8107760 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd53530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd5c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 0x7fa1980a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5578dccb7500 con 0x5578dcd5c650 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5578dcc5fbe0 con 0x5578dcd53a70 2024-03-16T08:19:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5578dccb35a0 con 0x5578dcd53150 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd53530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55190/0 (socket says 172.21.15.49:55190) 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 -- 172.21.15.49:0/1145281175 learned_addr learned my addr 172.21.15.49:0/1145281175 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd5c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 -- 172.21.15.49:0/1145281175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 msgr2=0x5578dcd53530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd53530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 -- 172.21.15.49:0/1145281175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 msgr2=0x5578dcd5c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd5c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.328+0000 7fa1ad463640 1 -- 172.21.15.49:0/1145281175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dcc734b0 con 0x5578dcd5c650 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd5c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 0x7fa1980a9240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa194000c00 tx=0x7fa19402c7e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0098580 tx=0x7fa1a009bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c354c316d65840da server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa192ffd640 1 -- 172.21.15.49:0/1145281175 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a009b6f0 con 0x5578dcd5c650 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa192ffd640 1 -- 172.21.15.49:0/1145281175 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a00051d0 con 0x5578dcd5c650 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.329+0000 7fa192ffd640 1 -- 172.21.15.49:0/1145281175 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00cf0b0 con 0x5578dcd5c650 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1145281175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 msgr2=0x5578dcd5ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0098580 tx=0x7fa1a009bc70 comp rx=0 tx=0).stop 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1145281175 shutdown_connections 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd5ea40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd5c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1145281175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd53530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1145281175 >> 172.21.15.49:0/1145281175 conn(0x5578dcd4d880 msgr2=0x5578dcd4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1145281175 shutdown_connections 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1145281175 wait complete. 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 -- start start 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.330+0000 7fa1b1b90740 1 Processor -- start 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53a70 0x5578dcd59b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd5c650 0x5578dcd5a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5578dcd5f440 con 0x5578dcd53150 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5578dcc73160 con 0x5578dcd5c650 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5578dcd5f5c0 con 0x5578dcd53a70 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 10 client.0 _ll_register_callbacks cb 0x5578dcc57150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55196/0 (socket says 172.21.15.49:55196) 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 -- 172.21.15.49:0/3712077089 learned_addr learned my addr 172.21.15.49:0/3712077089 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53a70 0x5578dcd59b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1ad463640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd5c650 0x5578dcd5a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 -- 172.21.15.49:0/3712077089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53a70 msgr2=0x5578dcd59b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53a70 0x5578dcd59b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 -- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd5c650 msgr2=0x5578dcd5a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd5c650 0x5578dcd5a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 -- 172.21.15.49:0/3712077089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dca957d0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1acc62640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa19c012770 tx=0x7fa19c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9462653d3ecf2b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa187fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00f920 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa19c00fac0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.331+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a00051d0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa19c00fcc0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 10 client.16122 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5578dc9cb3f0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a00051d0 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa19c01c420 con 0x5578dcd53150 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 10 client.16122 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 20 client.16122 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcd6c690 0x5578dcd6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 4 client.16122 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578dccc4a40 con 0x5578dcd6c690 2024-03-16T08:19:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.332+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcd6c690 0x5578dcd6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcd6c690 0x5578dcd6eb00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b906d5e7fb72580b server_cookie=7454f2e398da5fd5 in_seq=0 out_seq=0 2024-03-16T08:19:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa187fff640 10 client.16122 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa187fff640 1 -- 172.21.15.49:0/3712077089 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5578dccc4a40 con 0x5578dcd6c690 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa187fff640 10 client.16122 handle_command_reply: tid=0 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 client.16122 shutdown 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 msgr2=0x5578dcd59620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa19c012770 tx=0x7fa19c012c30 comp rx=0 tx=0).stop 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 shutdown_connections 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcd6c690 0x5578dcd6eb00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd5c650 0x5578dcd5a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa187fff640 0 client.16122 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53a70 0x5578dcd59b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/3712077089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd53150 0x5578dcd59620 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.333+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 >> 172.21.15.49:0/3712077089 conn(0x5578dcd4d880 msgr2=0x5578dcd4f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.334+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 shutdown_connections 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.334+0000 7fa1b1b90740 1 -- 172.21.15.49:0/3712077089 wait complete. 2024-03-16T08:19:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.334+0000 7fa1b1b90740 20 client.16122 trim_cache size 0 max 16384 2024-03-16T08:19:14.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 Processor -- start 2024-03-16T08:19:14.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 -- start start 2024-03-16T08:19:14.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd58560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81adac0 con 0x5578dcd5c650 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ab390 con 0x5578dcd53a70 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81a91c0 con 0x5578dcd53150 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd58560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55212/0 (socket says 172.21.15.49:55212) 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37576/0 (socket says 172.21.15.49:37576) 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 -- 172.21.15.49:0/1625561116 learned_addr learned my addr 172.21.15.49:0/1625561116 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 -- 172.21.15.49:0/1625561116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 msgr2=0x5578dcd5bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 -- 172.21.15.49:0/1625561116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 msgr2=0x5578dcd58560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd58560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1ad463640 1 -- 172.21.15.49:0/1625561116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dc98b0f0 con 0x5578dcd5c650 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1acc62640 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.353+0000 7fa1a7fff640 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd58560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.354+0000 7fa1ad463640 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa1a009bb40 tx=0x7fa1a00984b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cbaad09149c061 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.354+0000 7fa1927fc640 1 -- 172.21.15.49:0/1625561116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0098220 con 0x5578dcd5c650 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.354+0000 7fa1927fc640 1 -- 172.21.15.49:0/1625561116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0099eb0 con 0x5578dcd5c650 2024-03-16T08:19:14.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.354+0000 7fa1927fc640 1 -- 172.21.15.49:0/1625561116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a009b000 con 0x5578dcd5c650 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1625561116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 msgr2=0x5578dcd681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa1a009bb40 tx=0x7fa1a00984b0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1625561116 shutdown_connections 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd681e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd58560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/1625561116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1625561116 >> 172.21.15.49:0/1625561116 conn(0x5578dcd4d880 msgr2=0x5578dcd4f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1625561116 shutdown_connections 2024-03-16T08:19:14.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- 172.21.15.49:0/1625561116 wait complete. 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 -- start start 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.355+0000 7fa1b1b90740 1 Processor -- start 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd60840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81a9850 con 0x5578dcd5c650 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81aacc0 con 0x5578dcd53a70 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81a9f20 con 0x5578dcd53150 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1b1b90740 10 client.0 _ll_register_callbacks cb 0x7fa19c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1acc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd60300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37588/0 (socket says 172.21.15.49:37588) 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1acc62640 1 -- 172.21.15.49:0/2481467051 learned_addr learned my addr 172.21.15.49:0/2481467051 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd60840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55214/0 (socket says 172.21.15.49:55214) 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 -- 172.21.15.49:0/2481467051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 msgr2=0x5578dcd60300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd60300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 -- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 msgr2=0x5578dcd60840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd60840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 -- 172.21.15.49:0/2481467051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dcc7e9d0 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa1ad463640 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa1a009b8b0 tx=0x7fa1a00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460e6927a00faff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.356+0000 7fa187fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00ccf20 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa1a00cd0c0 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0099eb0 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa1a009b000 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 10 client.16128 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5578dcdd2d20 con 0x5578dcd5c650 2024-03-16T08:19:14.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0099eb0 con 0x5578dcd5c650 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa1a00a0c20 con 0x5578dcd5c650 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 10 client.16128 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 20 client.16128 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcdd2f40 0x5578dcdd5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 4 client.16128 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.357+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578dccbed90 con 0x5578dcdd2f40 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1acc62640 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcdd2f40 0x5578dcdd5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa187fff640 10 client.16128 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1acc62640 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcdd2f40 0x5578dcdd5330 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12e05e7c4562fff server_cookie=a0c6131e550c92c6 in_seq=0 out_seq=0 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa187fff640 1 -- 172.21.15.49:0/2481467051 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578dccbed90 con 0x5578dcdd2f40 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa187fff640 10 client.16128 handle_command_reply: tid=0 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 client.16128 shutdown 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 msgr2=0x5578dcd60d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa1a009b8b0 tx=0x7fa1a00058b0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 shutdown_connections 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5578dcdd2f40 0x5578dcdd5330 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578dcd5c650 0x5578dcd60d80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578dcd53a70 0x5578dcd60840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 --2- 172.21.15.49:0/2481467051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5578dcd53150 0x5578dcd60300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 >> 172.21.15.49:0/2481467051 conn(0x5578dcd4d880 msgr2=0x5578dcd595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.358+0000 7fa187fff640 0 client.16128 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 shutdown_connections 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1b1b90740 1 -- 172.21.15.49:0/2481467051 wait complete. 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1b1b90740 20 client.16128 trim_cache size 0 max 16384 2024-03-16T08:19:14.365 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:19:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:19:14.366 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:14.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 msgr2=0x7fa1980a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 0x7fa1980a9240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa194000c00 tx=0x7fa19402c7e0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 msgr2=0x7fa1a819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 0x7fa1a819c970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009fd0 tx=0x7fa1a000ef10 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 shutdown_connections 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa1980a6d80 0x7fa1980a9240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a8107760 0x7fa1a819c970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1a8069380 0x7fa1a81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 --2- 172.21.15.49:0/2095070576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1a80689c0 0x7fa1a81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 >> 172.21.15.49:0/2095070576 conn(0x7fa1a80777e0 msgr2=0x7fa1a80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.359+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 shutdown_connections 2024-03-16T08:19:14.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.360+0000 7fa1aeec6640 1 -- 172.21.15.49:0/2095070576 wait complete. 2024-03-16T08:19:14.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:13 smithi129 ceph-mon[36561]: pgmap v63: 97 pgs: 97 active+clean; 259 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-03-16T08:19:14.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:19:14.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 -- 172.21.15.49:0/907950363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541066b0 msgr2=0x7f0954106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 --2- 172.21.15.49:0/907950363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541066b0 0x7f0954106a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0944009930 tx=0x7f094402bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 -- 172.21.15.49:0/907950363 shutdown_connections 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 --2- 172.21.15.49:0/907950363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 0x7f095410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 --2- 172.21.15.49:0/907950363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541006a0 0x7f0954100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 --2- 172.21.15.49:0/907950363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541066b0 0x7f0954106a90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 -- 172.21.15.49:0/907950363 >> 172.21.15.49:0/907950363 conn(0x7f0954077780 msgr2=0x7f0954077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 -- 172.21.15.49:0/907950363 shutdown_connections 2024-03-16T08:19:14.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.531+0000 7f095a0f8640 1 -- 172.21.15.49:0/907950363 wait complete. 2024-03-16T08:19:14.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 Processor -- start 2024-03-16T08:19:14.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 -- start start 2024-03-16T08:19:14.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 0x7f0954199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 0x7f09541a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0954110a80 con 0x7f09541006a0 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0954110900 con 0x7f09541066b0 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f095a0f8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0954110c00 con 0x7f0954101060 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f09537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f09537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55220/0 (socket says 172.21.15.49:55220) 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f09537fe640 1 -- 172.21.15.49:0/2181111438 learned_addr learned my addr 172.21.15.49:0/2181111438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f0952ffd640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 0x7f0954199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.532+0000 7f0953fff640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 0x7f09541a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 -- 172.21.15.49:0/2181111438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 msgr2=0x7f0954199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 0x7f0954199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 -- 172.21.15.49:0/2181111438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 msgr2=0x7f09541a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 0x7f09541a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 -- 172.21.15.49:0/2181111438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09541a4500 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f0953fff640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 0x7f09541a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f09537fe640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0944036040 tx=0x7f0944004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944043020 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09541a46d0 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09440054e0 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094403e450 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.533+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541a4be0 con 0x7f09541006a0 2024-03-16T08:19:14.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.534+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0954101de0 con 0x7f09541006a0 2024-03-16T08:19:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.535+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0944046020 con 0x7f09541006a0 2024-03-16T08:19:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.536+0000 7f0950ff9640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 0x7f09400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.536+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f09440f1690 con 0x7f09541006a0 2024-03-16T08:19:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.536+0000 7f0952ffd640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 0x7f09400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.537+0000 7f0952ffd640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 0x7f09400a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f095406a960 tx=0x7f093c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.540+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f094404d020 con 0x7f09541006a0 2024-03-16T08:19:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.796+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0954068a20 con 0x7f09541006a0 2024-03-16T08:19:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.797+0000 7f0950ff9640 1 -- 172.21.15.49:0/2181111438 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f0954068a20 con 0x7f09541006a0 2024-03-16T08:19:14.798 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:19:14.798 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:19:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 msgr2=0x7f09400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 0x7f09400a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f095406a960 tx=0x7f093c005f90 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 msgr2=0x7f0954198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0944036040 tx=0x7f0944004040 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 shutdown_connections 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f09400a2970 0x7f09400a4e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09541066b0 0x7f09541a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0954101060 0x7f0954199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 --2- 172.21.15.49:0/2181111438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09541006a0 0x7f0954198f40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 >> 172.21.15.49:0/2181111438 conn(0x7f0954077780 msgr2=0x7f09540fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.799+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 shutdown_connections 2024-03-16T08:19:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.800+0000 7f095a0f8640 1 -- 172.21.15.49:0/2181111438 wait complete. 2024-03-16T08:19:14.812 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 -- 172.21.15.49:0/840281543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 msgr2=0x7f49f4107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 --2- 172.21.15.49:0/840281543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f4107b40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009930 tx=0x7f49e402bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 -- 172.21.15.49:0/840281543 shutdown_connections 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 --2- 172.21.15.49:0/840281543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 --2- 172.21.15.49:0/840281543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 0x7f49f4068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 --2- 172.21.15.49:0/840281543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f4107b40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 -- 172.21.15.49:0/840281543 >> 172.21.15.49:0/840281543 conn(0x7f49f40777e0 msgr2=0x7f49f4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 -- 172.21.15.49:0/840281543 shutdown_connections 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.933+0000 7f49f96b2640 1 -- 172.21.15.49:0/840281543 wait complete. 2024-03-16T08:19:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 Processor -- start 2024-03-16T08:19:14.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 -- start start 2024-03-16T08:19:14.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 0x7f49f41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4111b70 con 0x7f49f4107760 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41119f0 con 0x7f49f4069380 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f96b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4111cf0 con 0x7f49f40689c0 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55238/0 (socket says 172.21.15.49:55238) 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f37fe640 1 -- 172.21.15.49:0/830154472 learned_addr learned my addr 172.21.15.49:0/830154472 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f27fc640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.934+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 0x7f49f41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 -- 172.21.15.49:0/830154472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 msgr2=0x7f49f41980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 0x7f49f41980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 -- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 msgr2=0x7f49f419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 -- 172.21.15.49:0/830154472 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f419d0c0 con 0x7f49f4069380 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f37fe640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f27fc640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f4198610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9f0 tx=0x7f49e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49dbfff640 1 -- 172.21.15.49:0/830154472 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800ccc0 con 0x7f49f4069380 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49dbfff640 1 -- 172.21.15.49:0/830154472 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8005020 con 0x7f49f4069380 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49dbfff640 1 -- 172.21.15.49:0/830154472 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80049e0 con 0x7f49f4069380 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f419d3b0 con 0x7f49f4069380 2024-03-16T08:19:14.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.935+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a4c40 con 0x7f49f4069380 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 Processor -- start 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 -- start start 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49dbfff640 1 -- 172.21.15.49:0/830154472 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f49e8005560 con 0x7f49f4069380 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fe150 0x55928aa067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa09120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55928a961490 con 0x55928a9fe150 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55928a909c90 con 0x55928aa06d30 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49fc379740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55928a95d530 con 0x55928a9fd830 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37638/0 (socket says 172.21.15.49:37638) 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49f37fe640 1 -- 172.21.15.49:0/710451075 learned_addr learned my addr 172.21.15.49:0/710451075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fe150 0x55928aa067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.938+0000 7f49f27fc640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa09120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 -- 172.21.15.49:0/710451075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 msgr2=0x55928aa09120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa09120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 -- 172.21.15.49:0/710451075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fe150 msgr2=0x55928aa067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fe150 0x55928aa067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 -- 172.21.15.49:0/710451075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928a9c4500 con 0x55928a9fd830 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f27fc640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa09120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49dbfff640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 0x7f49dc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f37fe640 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49ec011850 tx=0x7f49ec011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9718d66963214b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 0x7f49dc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49dbfff640 1 -- 172.21.15.49:0/830154472 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f49e80cede0 con 0x7f49f4069380 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49d97fa640 1 -- 172.21.15.49:0/710451075 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec011330 con 0x55928a9fd830 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49d97fa640 1 -- 172.21.15.49:0/710451075 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec00fe80 con 0x55928a9fd830 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.939+0000 7f49d97fa640 1 -- 172.21.15.49:0/710451075 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0193e0 con 0x55928a9fd830 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 0x7f49dc0a4e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005ab0 tx=0x7f49e402e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- 172.21.15.49:0/710451075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 msgr2=0x55928a9fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49ec011850 tx=0x7f49ec011d10 comp rx=0 tx=0).stop 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- 172.21.15.49:0/710451075 shutdown_connections 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa09120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fe150 0x55928aa067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 --2- 172.21.15.49:0/710451075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928a9fdc10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- 172.21.15.49:0/710451075 >> 172.21.15.49:0/710451075 conn(0x55928a9f7f60 msgr2=0x55928a9f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- 172.21.15.49:0/710451075 shutdown_connections 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- 172.21.15.49:0/710451075 wait complete. 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 -- start start 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.940+0000 7f49fc379740 1 Processor -- start 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 0x55928aa04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55928aa09b20 con 0x55928aa06d30 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55928a91d230 con 0x55928a9fe150 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55928aa09ca0 con 0x55928a9fd830 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 0x55928aa04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49fc379740 10 client.0 _ll_register_callbacks cb 0x55928a959010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37654/0 (socket says 172.21.15.49:37654) 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f2ffd640 1 -- 172.21.15.49:0/4070664318 learned_addr learned my addr 172.21.15.49:0/4070664318 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 0x55928aa04220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39762/0 (socket says 172.21.15.49:39762) 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 -- 172.21.15.49:0/4070664318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 msgr2=0x55928aa03ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 -- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 msgr2=0x55928aa04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 0x55928aa04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f37fe640 1 -- 172.21.15.49:0/4070664318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928a92c100 con 0x55928aa06d30 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.941+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49f37fe640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa04760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00f790 tx=0x7f49ec011130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b91f8d98a8cde6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec01f040 con 0x55928aa06d30 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f49ec0113e0 con 0x55928aa06d30 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55928a96e9d0 con 0x55928aa06d30 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f49ec004a40 con 0x55928aa06d30 2024-03-16T08:19:14.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49fc379740 10 client.16146 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55928a6753f0 con 0x55928aa06d30 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55928a96e9d0 con 0x55928aa06d30 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.942+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f49ec018070 con 0x55928aa06d30 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 10 client.16146 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 20 client.16146 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa170f0 0x55928aa194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 4 client.16146 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928a968d20 con 0x55928aa170f0 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa170f0 0x55928aa194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa170f0 0x55928aa194e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad7515a58deeb174 server_cookie=8f1edaa328cefa77 in_seq=0 out_seq=0 2024-03-16T08:19:14.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.943+0000 7f49d6ff5640 10 client.16146 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:14.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49d6ff5640 1 -- 172.21.15.49:0/4070664318 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55928a968d20 con 0x55928aa170f0 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49d6ff5640 10 client.16146 handle_command_reply: tid=0 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 client.16146 shutdown 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 msgr2=0x55928aa04760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa04760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00f790 tx=0x7f49ec011130 comp rx=0 tx=0).stop 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 shutdown_connections 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa170f0 0x55928aa194e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa04760 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fe150 0x55928aa04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49fc379740 1 --2- 172.21.15.49:0/4070664318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fd830 0x55928aa03ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.944+0000 7f49d6ff5640 0 client.16146 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.945+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 >> 172.21.15.49:0/4070664318 conn(0x55928a9f7f60 msgr2=0x55928a9f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.945+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 shutdown_connections 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.945+0000 7f49fc379740 1 -- 172.21.15.49:0/4070664318 wait complete. 2024-03-16T08:19:14.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.945+0000 7f49fc379740 20 client.16146 trim_cache size 0 max 16384 2024-03-16T08:19:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 Processor -- start 2024-03-16T08:19:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 -- start start 2024-03-16T08:19:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa03490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa05b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41afd90 con 0x55928a9fd830 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ad660 con 0x55928aa06d30 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49fc379740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ab490 con 0x55928a9fe150 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55254/0 (socket says 172.21.15.49:55254) 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49f2ffd640 1 -- 172.21.15.49:0/2134946589 learned_addr learned my addr 172.21.15.49:0/2134946589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.960+0000 7f49f27fc640 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa03490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 -- 172.21.15.49:0/2134946589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 msgr2=0x55928aa03490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa03490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 -- 172.21.15.49:0/2134946589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 msgr2=0x55928aa128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 -- 172.21.15.49:0/2134946589 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928a73f7d0 con 0x55928aa06d30 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49f37fe640 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa05b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0054e0 tx=0x7f49ec009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bdc52964bf3413c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49d8ff9640 1 -- 172.21.15.49:0/2134946589 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f7c0 con 0x55928aa06d30 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49d8ff9640 1 -- 172.21.15.49:0/2134946589 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec0113e0 con 0x55928aa06d30 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.961+0000 7f49d8ff9640 1 -- 172.21.15.49:0/2134946589 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005890 con 0x55928aa06d30 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- 172.21.15.49:0/2134946589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 msgr2=0x55928aa05b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa05b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0054e0 tx=0x7f49ec009230 comp rx=0 tx=0).stop 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- 172.21.15.49:0/2134946589 shutdown_connections 2024-03-16T08:19:14.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928aa06d30 0x55928aa05b10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa03490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 --2- 172.21.15.49:0/2134946589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928a9fd830 0x55928aa128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- 172.21.15.49:0/2134946589 >> 172.21.15.49:0/2134946589 conn(0x55928a9f7f60 msgr2=0x55928aa05400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- 172.21.15.49:0/2134946589 shutdown_connections 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- 172.21.15.49:0/2134946589 wait complete. 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 -- start start 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 Processor -- start 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.962+0000 7f49fc379740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41abb20 con 0x55928aa06d30 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41acf90 con 0x55928a9fd830 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ac1f0 con 0x55928a9fe150 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 10 client.0 _ll_register_callbacks cb 0x7f49ec00fdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:14.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55258/0 (socket says 172.21.15.49:55258) 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39780/0 (socket says 172.21.15.49:39780) 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 -- 172.21.15.49:0/3638865870 learned_addr learned my addr 172.21.15.49:0/3638865870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 -- 172.21.15.49:0/3638865870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 msgr2=0x55928aa0a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa0a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 -- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 msgr2=0x55928aa09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 -- 172.21.15.49:0/3638865870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928aa7b390 con 0x55928aa06d30 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49f37fe640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005a00 tx=0x7f49ec005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c18cbf6cf8cc60f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:14.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec03a060 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f49ec005000 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.963+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ec0113e0 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f49ec00f7c0 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 10 client.16152 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55928aa7c690 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec0113e0 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f49ec034070 con 0x55928aa06d30 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 10 client.16152 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 10 client.16152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 20 client.16152 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa7cca0 0x55928aa7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 4 client.16152 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:14.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55928a8e66a0 con 0x55928aa7cca0 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.964+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa7cca0 0x55928aa7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49d6ff5640 10 client.16152 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49f2ffd640 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa7cca0 0x55928aa7f0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52f7f1d55c6d83eb server_cookie=124d00a7b1983b57 in_seq=0 out_seq=0 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49d6ff5640 1 -- 172.21.15.49:0/3638865870 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55928a8e66a0 con 0x55928aa7cca0 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49d6ff5640 10 client.16152 handle_command_reply: tid=0 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 client.16152 shutdown 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 msgr2=0x55928aa0a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005a00 tx=0x7f49ec005a30 comp rx=0 tx=0).stop 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 shutdown_connections 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55928aa7cca0 0x55928aa7f0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55928aa06d30 0x55928aa0a970 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49d6ff5640 0 client.16152 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928a9fe150 0x55928aa0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 --2- 172.21.15.49:0/3638865870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55928a9fd830 0x55928aa09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 >> 172.21.15.49:0/3638865870 conn(0x55928a9f7f60 msgr2=0x55928aa05bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 shutdown_connections 2024-03-16T08:19:14.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.965+0000 7f49fc379740 1 -- 172.21.15.49:0/3638865870 wait complete. 2024-03-16T08:19:14.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49fc379740 20 client.16152 trim_cache size 0 max 16384 2024-03-16T08:19:14.973 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 msgr2=0x7f49dc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 0x7f49dc0a4e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005ab0 tx=0x7f49e402e040 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 msgr2=0x7f49f4198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f4198610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9f0 tx=0x7f49e800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 shutdown_connections 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f49dc0a2970 0x7f49dc0a4e30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49f4107760 0x7f49f419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f4069380 0x7f49f4198610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 --2- 172.21.15.49:0/830154472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49f40689c0 0x7f49f41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:14.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 >> 172.21.15.49:0/830154472 conn(0x7f49f40777e0 msgr2=0x7f49f40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:14.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 shutdown_connections 2024-03-16T08:19:14.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:14.966+0000 7f49f96b2640 1 -- 172.21.15.49:0/830154472 wait complete. 2024-03-16T08:19:14.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:19:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- 172.21.15.49:0/3872388653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3060107830 msgr2=0x7f3060107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 --2- 172.21.15.49:0/3872388653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3060107830 0x7f3060107c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f305c009930 tx=0x7f305c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- 172.21.15.49:0/3872388653 shutdown_connections 2024-03-16T08:19:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 --2- 172.21.15.49:0/3872388653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30600693a0 0x7f306010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 --2- 172.21.15.49:0/3872388653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f3060068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 --2- 172.21.15.49:0/3872388653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3060107830 0x7f3060107c10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- 172.21.15.49:0/3872388653 >> 172.21.15.49:0/3872388653 conn(0x7f30600777e0 msgr2=0x7f3060077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- 172.21.15.49:0/3872388653 shutdown_connections 2024-03-16T08:19:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- 172.21.15.49:0/3872388653 wait complete. 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 Processor -- start 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.102+0000 7f3067436640 1 -- start start 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f30601006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3060107830 0x7f3060101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3060111b40 con 0x7f30600693a0 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30601119c0 con 0x7f30600689e0 2024-03-16T08:19:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f3067436640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3060111cc0 con 0x7f3060107830 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30651d2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f30601006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55282/0 (socket says 172.21.15.49:55282) 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 -- 172.21.15.49:0/1079748728 learned_addr learned my addr 172.21.15.49:0/1079748728 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30659d3640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3060107830 0x7f3060101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 -- 172.21.15.49:0/1079748728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3060107830 msgr2=0x7f3060101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3060107830 0x7f3060101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 -- 172.21.15.49:0/1079748728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 msgr2=0x7f30601006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f30601006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30649d1640 1 -- 172.21.15.49:0/1079748728 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601a6520 con 0x7f30600693a0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.103+0000 7f30651d2640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f30601006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f30649d1640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f305000e9f0 tx=0x7f305000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f30567fc640 1 -- 172.21.15.49:0/1079748728 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305000cd80 con 0x7f30600693a0 2024-03-16T08:19:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601a6810 con 0x7f30600693a0 2024-03-16T08:19:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f30567fc640 1 -- 172.21.15.49:0/1079748728 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050010910 con 0x7f30600693a0 2024-03-16T08:19:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f30567fc640 1 -- 172.21.15.49:0/1079748728 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500188f0 con 0x7f30600693a0 2024-03-16T08:19:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.104+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601a6db0 con 0x7f30600693a0 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30567fc640 1 -- 172.21.15.49:0/1079748728 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3050010ac0 con 0x7f30600693a0 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 Processor -- start 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 -- start start 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c39753b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 0x55c39753e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c397496490 con 0x55c39753bd30 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c39743ec90 con 0x55c397532830 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f306a025740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c397492530 con 0x55c397533150 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30659d3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30649d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 0x55c39753e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30659d3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39804/0 (socket says 172.21.15.49:39804) 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30659d3640 1 -- 172.21.15.49:0/1461326102 learned_addr learned my addr 172.21.15.49:0/1461326102 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.107+0000 7f30651d2640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c39753b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30567fc640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 0x7f304c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 -- 172.21.15.49:0/1461326102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 msgr2=0x55c397532c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 -- 172.21.15.49:0/1461326102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 msgr2=0x55c39753e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 0x55c39753e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 -- 172.21.15.49:0/1461326102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3974f9500 con 0x55c397533150 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30567fc640 1 -- 172.21.15.49:0/1079748728 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f30500cf1e0 con 0x7f30600693a0 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30659d3640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30649d1640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 0x55c39753e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30659d3640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 0x7f304c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f30651d2640 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c39753b7f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f305c009230 tx=0x7f305c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5775dbe29797d748 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f304b7fe640 1 -- 172.21.15.49:0/1461326102 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c042070 con 0x55c397533150 2024-03-16T08:19:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f304b7fe640 1 -- 172.21.15.49:0/1461326102 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c004a70 con 0x55c397533150 2024-03-16T08:19:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.108+0000 7f304b7fe640 1 -- 172.21.15.49:0/1461326102 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c005150 con 0x55c397533150 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 -- 172.21.15.49:0/1461326102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 msgr2=0x55c39753b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f30659d3640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 0x7f304c0a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30580107d0 tx=0x7f3058004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c39753b7f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f305c009230 tx=0x7f305c005d50 comp rx=0 tx=0).stop 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 -- 172.21.15.49:0/1461326102 shutdown_connections 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c39753bd30 0x55c39753e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c39753b7f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 --2- 172.21.15.49:0/1461326102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c397532c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 -- 172.21.15.49:0/1461326102 >> 172.21.15.49:0/1461326102 conn(0x55c39752cf60 msgr2=0x55c39752d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 -- 172.21.15.49:0/1461326102 shutdown_connections 2024-03-16T08:19:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.109+0000 7f306a025740 1 -- 172.21.15.49:0/1461326102 wait complete. 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 -- start start 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 Processor -- start 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c39753eac0 con 0x55c397532830 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c397452230 con 0x55c397533150 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c39753ec40 con 0x55c39753bd30 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f306a025740 10 client.0 _ll_register_callbacks cb 0x55c39748e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f30651d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.110+0000 7f30649d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30651d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55294/0 (socket says 172.21.15.49:55294) 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30649d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39812/0 (socket says 172.21.15.49:39812) 2024-03-16T08:19:15.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30651d2640 1 -- 172.21.15.49:0/1336832948 learned_addr learned my addr 172.21.15.49:0/1336832948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37690/0 (socket says 172.21.15.49:37690) 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 -- 172.21.15.49:0/1336832948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 msgr2=0x55c3975392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 -- 172.21.15.49:0/1336832948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 msgr2=0x55c397538db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 -- 172.21.15.49:0/1336832948 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c397461100 con 0x55c39753bd30 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30649d1640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30651d2640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f30659d3640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f305800f790 tx=0x7f3058004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff2337fcc5e177b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f3048ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004d20 con 0x55c39753bd30 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3058005020 con 0x55c39753bd30 2024-03-16T08:19:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.111+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c004a70 con 0x55c39753bd30 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f305801fe10 con 0x55c39753bd30 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 10 client.26044 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c3971aa3f0 con 0x55c39753bd30 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c004a70 con 0x55c39753bd30 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3058026030 con 0x55c39753bd30 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 10 client.26044 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 10 client.26044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 20 client.26044 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c39754c050 0x55c39754e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 4 client.26044 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3974a39d0 con 0x55c39754c050 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.112+0000 7f30651d2640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c39754c050 0x55c39754e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.113+0000 7f3048ff9640 10 client.26044 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.113+0000 7f30651d2640 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c39754c050 0x55c39754e460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d5d69c84a4da23 server_cookie=7a3ca51ede085d97 in_seq=0 out_seq=0 2024-03-16T08:19:15.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.113+0000 7f3048ff9640 1 -- 172.21.15.49:0/1336832948 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c3974a39d0 con 0x55c39754c050 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.113+0000 7f3048ff9640 10 client.26044 handle_command_reply: tid=0 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 client.26044 shutdown 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 msgr2=0x55c397539830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f305800f790 tx=0x7f3058004ab0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 shutdown_connections 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c39754c050 0x55c39754e460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397539830 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397533150 0x55c3975392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f3048ff9640 0 client.26044 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 --2- 172.21.15.49:0/1336832948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c397538db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 >> 172.21.15.49:0/1336832948 conn(0x55c39752cf60 msgr2=0x55c39752ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 shutdown_connections 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 1 -- 172.21.15.49:0/1336832948 wait complete. 2024-03-16T08:19:15.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.114+0000 7f306a025740 20 client.26044 trim_cache size 0 max 16384 2024-03-16T08:19:15.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 Processor -- start 2024-03-16T08:19:15.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 -- start start 2024-03-16T08:19:15.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c397547860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39753bd30 0x55c39752be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30601b1ea0 con 0x55c397532830 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30601af770 con 0x55c39753bd30 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f306a025740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30601ad5a0 con 0x55c397533150 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f30651d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f30649d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c397547860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f30651d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55310/0 (socket says 172.21.15.49:55310) 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f30659d3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39753bd30 0x55c39752be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.128+0000 7f30651d2640 1 -- 172.21.15.49:0/3899833722 learned_addr learned my addr 172.21.15.49:0/3899833722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 -- 172.21.15.49:0/3899833722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 msgr2=0x55c397547860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c397547860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 -- 172.21.15.49:0/3899833722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39753bd30 msgr2=0x55c39752be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39753bd30 0x55c39752be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 -- 172.21.15.49:0/3899833722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3972747d0 con 0x55c397532830 2024-03-16T08:19:15.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f30651d2640 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f305c009230 tx=0x7f305c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c739a0c017222d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f304affd640 1 -- 172.21.15.49:0/3899833722 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c042040 con 0x55c397532830 2024-03-16T08:19:15.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f304affd640 1 -- 172.21.15.49:0/3899833722 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c004a70 con 0x55c397532830 2024-03-16T08:19:15.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.129+0000 7f304affd640 1 -- 172.21.15.49:0/3899833722 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c005400 con 0x55c397532830 2024-03-16T08:19:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- 172.21.15.49:0/3899833722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 msgr2=0x55c39753b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f305c009230 tx=0x7f305c005fd0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- 172.21.15.49:0/3899833722 shutdown_connections 2024-03-16T08:19:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39753bd30 0x55c39752be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c397533150 0x55c397547860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 --2- 172.21.15.49:0/3899833722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397532830 0x55c39753b4e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- 172.21.15.49:0/3899833722 >> 172.21.15.49:0/3899833722 conn(0x55c39752cf60 msgr2=0x55c39752b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- 172.21.15.49:0/3899833722 shutdown_connections 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- 172.21.15.49:0/3899833722 wait complete. 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 -- start start 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.130+0000 7f306a025740 1 Processor -- start 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c3975357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397536270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30601adc30 con 0x55c397533150 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30601af0a0 con 0x55c397532830 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30601ae300 con 0x55c39753bd30 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 10 client.0 _ll_register_callbacks cb 0x7f3058005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55318/0 (socket says 172.21.15.49:55318) 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 -- 172.21.15.49:0/3189515105 learned_addr learned my addr 172.21.15.49:0/3189515105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30651d2640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c3975357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30659d3640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397536270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 -- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 msgr2=0x55c397536270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397536270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 -- 172.21.15.49:0/3189515105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 msgr2=0x55c3975357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c3975357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 -- 172.21.15.49:0/3189515105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3975b1360 con 0x55c397533150 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30651d2640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c3975357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f30649d1640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f305009b8d0 tx=0x7f3050009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f42bdfc5083d444 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f3048ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305009b940 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f305009cbc0 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.131+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c004a70 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f305009f9e0 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 10 client.16170 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c3975b2590 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c004a70 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3050097890 con 0x55c397533150 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 10 client.16170 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 10 client.16170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 20 client.16170 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c3975b2c10 0x55c3975b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 4 client.16170 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c39749dd20 con 0x55c3975b2c10 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.132+0000 7f30651d2640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c3975b2c10 0x55c3975b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f30651d2640 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c3975b2c10 0x55c3975b5000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0f7e60d82642b1e server_cookie=13a9df039c399755 in_seq=0 out_seq=0 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f3048ff9640 10 client.16170 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f3048ff9640 1 -- 172.21.15.49:0/3189515105 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c39749dd20 con 0x55c3975b2c10 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f3048ff9640 10 client.16170 handle_command_reply: tid=0 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 client.16170 shutdown 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 msgr2=0x55c397535d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f305009b8d0 tx=0x7f3050009310 comp rx=0 tx=0).stop 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 shutdown_connections 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55c3975b2c10 0x55c3975b5000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c39753bd30 0x55c397536270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f3048ff9640 0 client.16170 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c397533150 0x55c397535d30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 --2- 172.21.15.49:0/3189515105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c397532830 0x55c3975357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 >> 172.21.15.49:0/3189515105 conn(0x55c39752cf60 msgr2=0x55c397539600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 shutdown_connections 2024-03-16T08:19:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 1 -- 172.21.15.49:0/3189515105 wait complete. 2024-03-16T08:19:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.133+0000 7f306a025740 20 client.16170 trim_cache size 0 max 16384 2024-03-16T08:19:15.141 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 msgr2=0x7f304c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 0x7f304c0a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30580107d0 tx=0x7f3058004a40 comp rx=0 tx=0).stop 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 msgr2=0x7f3060100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f305000e9f0 tx=0x7f305000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 shutdown_connections 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f304c0a2970 0x7f304c0a4e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3060107830 0x7f3060101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30600693a0 0x7f3060100be0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 --2- 172.21.15.49:0/1079748728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30600689e0 0x7f30601006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 >> 172.21.15.49:0/1079748728 conn(0x7f30600777e0 msgr2=0x7f30600fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 shutdown_connections 2024-03-16T08:19:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.134+0000 7f3067436640 1 -- 172.21.15.49:0/1079748728 wait complete. 2024-03-16T08:19:15.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:14 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2181111438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:15.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:19:15.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:14 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2181111438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:15.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 -- 172.21.15.49:0/947674117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50100650 msgr2=0x7fef50100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 --2- 172.21.15.49:0/947674117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50100650 0x7fef50100ad0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009930 tx=0x7fef3c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 -- 172.21.15.49:0/947674117 shutdown_connections 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 --2- 172.21.15.49:0/947674117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50101010 0x7fef5010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 --2- 172.21.15.49:0/947674117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50100650 0x7fef50100ad0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 --2- 172.21.15.49:0/947674117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50106660 0x7fef50106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 -- 172.21.15.49:0/947674117 >> 172.21.15.49:0/947674117 conn(0x7fef500777e0 msgr2=0x7fef50077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 -- 172.21.15.49:0/947674117 shutdown_connections 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.270+0000 7fef56b53640 1 -- 172.21.15.49:0/947674117 wait complete. 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 Processor -- start 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 -- start start 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 0x7fef501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50106660 0x7fef5019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef50110a30 con 0x7fef50101010 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef501108b0 con 0x7fef50106660 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef56b53640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef50110bb0 con 0x7fef50100650 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55334/0 (socket says 172.21.15.49:55334) 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 -- 172.21.15.49:0/381390482 learned_addr learned my addr 172.21.15.49:0/381390482 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef548ef640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 0x7fef501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef550f0640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50106660 0x7fef5019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 -- 172.21.15.49:0/381390482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 msgr2=0x7fef501980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 0x7fef501980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 -- 172.21.15.49:0/381390482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50106660 msgr2=0x7fef5019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50106660 0x7fef5019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 -- 172.21.15.49:0/381390482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5019d090 con 0x7fef50101010 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef548ef640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 0x7fef501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.271+0000 7fef4ffff640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009230 tx=0x7fef3c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.272+0000 7fef4dffb640 1 -- 172.21.15.49:0/381390482 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c041070 con 0x7fef50101010 2024-03-16T08:19:15.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.272+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5019d320 con 0x7fef50101010 2024-03-16T08:19:15.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.272+0000 7fef4dffb640 1 -- 172.21.15.49:0/381390482 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef3c004450 con 0x7fef50101010 2024-03-16T08:19:15.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.272+0000 7fef4dffb640 1 -- 172.21.15.49:0/381390482 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003e50 con 0x7fef50101010 2024-03-16T08:19:15.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.272+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a4c10 con 0x7fef50101010 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.274+0000 7fef4dffb640 1 -- 172.21.15.49:0/381390482 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fef3c044070 con 0x7fef50101010 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 Processor -- start 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 -- start start 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef4dffb640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 0x7fef400a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 0x561dc4ab27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 0x561dc4ab5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561dc4a0d490 con 0x561dc4ab2d30 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dc49b5c90 con 0x561dc4aaa150 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef5981c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561dc4a09530 con 0x561dc4aa9830 2024-03-16T08:19:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 0x561dc4ab27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef550f0640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 0x7fef400a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef548ef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef548ef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37756/0 (socket says 172.21.15.49:37756) 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 0x561dc4ab27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39864/0 (socket says 172.21.15.49:39864) 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef548ef640 1 -- 172.21.15.49:0/2977515942 learned_addr learned my addr 172.21.15.49:0/2977515942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef4dffb640 1 -- 172.21.15.49:0/381390482 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fef3c0f05c0 con 0x7fef50101010 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.275+0000 7fef550f0640 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 0x561dc4ab5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 -- 172.21.15.49:0/2977515942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 msgr2=0x561dc4ab27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 0x561dc4ab27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 -- 172.21.15.49:0/2977515942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 msgr2=0x561dc4ab5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 0x561dc4ab5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 -- 172.21.15.49:0/2977515942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc4a70500 con 0x561dc4aa9830 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef550f0640 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 0x561dc4ab5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:19:15.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef548ef640 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef44013800 tx=0x7fef44013cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d4990567c5d542 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef550f0640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 0x7fef400a4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x561dc4aa50f0 tx=0x7fef48009b00 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef3affd640 1 -- 172.21.15.49:0/2977515942 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44021070 con 0x561dc4aa9830 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef3affd640 1 -- 172.21.15.49:0/2977515942 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4401c410 con 0x561dc4aa9830 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.276+0000 7fef3affd640 1 -- 172.21.15.49:0/2977515942 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4401b5d0 con 0x561dc4aa9830 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- 172.21.15.49:0/2977515942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 msgr2=0x561dc4aa9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef44013800 tx=0x7fef44013cc0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- 172.21.15.49:0/2977515942 shutdown_connections 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4ab2d30 0x561dc4ab5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aaa150 0x561dc4ab27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 --2- 172.21.15.49:0/2977515942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aa9c10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- 172.21.15.49:0/2977515942 >> 172.21.15.49:0/2977515942 conn(0x561dc4aa3f60 msgr2=0x561dc4aa4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- 172.21.15.49:0/2977515942 shutdown_connections 2024-03-16T08:19:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- 172.21.15.49:0/2977515942 wait complete. 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.277+0000 7fef5981c740 1 -- start start 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 Processor -- start 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4ab0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561dc4ab5b20 con 0x561dc4aaa150 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dc49c9230 con 0x561dc4ab2d30 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561dc4ab5ca0 con 0x561dc4aa9830 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef5981c740 10 client.0 _ll_register_callbacks cb 0x561dc4a05010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef548ef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef550f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4ab0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55356/0 (socket says 172.21.15.49:55356) 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef548ef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aafce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37764/0 (socket says 172.21.15.49:37764) 2024-03-16T08:19:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.278+0000 7fef4ffff640 1 -- 172.21.15.49:0/872996376 learned_addr learned my addr 172.21.15.49:0/872996376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 -- 172.21.15.49:0/872996376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 msgr2=0x561dc4aafce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aafce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 -- 172.21.15.49:0/872996376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 msgr2=0x561dc4ab0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 -- 172.21.15.49:0/872996376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc49d8100 con 0x561dc4ab2d30 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef4ffff640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef550f0640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4ab0760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef480109a0 tx=0x7fef48004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98850cfa2c1c57e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:19:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48024030 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fef48009040 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef4401c410 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef48030050 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef5981c740 10 client.26060 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.279+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561dc47213f0 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4401c410 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef480283c0 con 0x561dc4ab2d30 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 10 client.26060 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 20 client.26060 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4ac2fa0 0x561dc4ac53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 4 client.26060 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc4a1a9d0 con 0x561dc4ac2fa0 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.280+0000 7fef548ef640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4ac2fa0 0x561dc4ac53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.281+0000 7fef2ffff640 10 client.26060 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.281+0000 7fef548ef640 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4ac2fa0 0x561dc4ac53d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a601de32f4e4f2 server_cookie=c38a52704e2e7d70 in_seq=0 out_seq=0 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.281+0000 7fef2ffff640 1 -- 172.21.15.49:0/872996376 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561dc4a1a9d0 con 0x561dc4ac2fa0 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.281+0000 7fef2ffff640 10 client.26060 handle_command_reply: tid=0 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 client.26060 shutdown 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 msgr2=0x561dc4ab0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4ab0760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef480109a0 tx=0x7fef48004a40 comp rx=0 tx=0).stop 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 shutdown_connections 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4ac2fa0 0x561dc4ac53d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4ab0760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4ab0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 --2- 172.21.15.49:0/872996376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aa9830 0x561dc4aafce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 >> 172.21.15.49:0/872996376 conn(0x561dc4aa3f60 msgr2=0x561dc4aa5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef2ffff640 0 client.26060 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 shutdown_connections 2024-03-16T08:19:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 1 -- 172.21.15.49:0/872996376 wait complete. 2024-03-16T08:19:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.282+0000 7fef5981c740 20 client.26060 trim_cache size 0 max 16384 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 Processor -- start 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 -- start start 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4a9ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 0x561dc4abe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef501adac0 con 0x561dc4aaa150 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ab390 con 0x561dc4aa9830 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef5981c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a91e0 con 0x561dc4ab2d30 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39888/0 (socket says 172.21.15.49:39888) 2024-03-16T08:19:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 -- 172.21.15.49:0/2136984740 learned_addr learned my addr 172.21.15.49:0/2136984740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef4ffff640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4a9ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef550f0640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 0x561dc4abe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 -- 172.21.15.49:0/2136984740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 msgr2=0x561dc4abe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 0x561dc4abe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 -- 172.21.15.49:0/2136984740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 msgr2=0x561dc4a9ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4a9ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef548ef640 1 -- 172.21.15.49:0/2136984740 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc4a9f640 con 0x561dc4aa9830 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.298+0000 7fef550f0640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 0x561dc4abe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.299+0000 7fef548ef640 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef44009fd0 tx=0x7fef4400e830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f92fa9096b66c05 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.299+0000 7fef3a7fc640 1 -- 172.21.15.49:0/2136984740 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44021040 con 0x561dc4aa9830 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.299+0000 7fef3a7fc640 1 -- 172.21.15.49:0/2136984740 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44031070 con 0x561dc4aa9830 2024-03-16T08:19:15.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.299+0000 7fef3a7fc640 1 -- 172.21.15.49:0/2136984740 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4400eb90 con 0x561dc4aa9830 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- 172.21.15.49:0/2136984740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 msgr2=0x561dc4ab2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef44009fd0 tx=0x7fef4400e830 comp rx=0 tx=0).stop 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- 172.21.15.49:0/2136984740 shutdown_connections 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4ab2d30 0x561dc4abe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aaa150 0x561dc4a9ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 --2- 172.21.15.49:0/2136984740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4aa9830 0x561dc4ab2410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- 172.21.15.49:0/2136984740 >> 172.21.15.49:0/2136984740 conn(0x561dc4aa3f60 msgr2=0x561dc4aae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- 172.21.15.49:0/2136984740 shutdown_connections 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- 172.21.15.49:0/2136984740 wait complete. 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 -- start start 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.300+0000 7fef5981c740 1 Processor -- start 2024-03-16T08:19:15.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aaa150 0x561dc4aabf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4aac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a9850 con 0x561dc4aa9830 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef501aacc0 con 0x561dc4ab2d30 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a9f20 con 0x561dc4aaa150 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef5981c740 10 client.0 _ll_register_callbacks cb 0x7fef48010650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55368/0 (socket says 172.21.15.49:55368) 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 -- 172.21.15.49:0/550245968 learned_addr learned my addr 172.21.15.49:0/550245968 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef4ffff640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aaa150 0x561dc4aabf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef550f0640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4aac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 -- 172.21.15.49:0/550245968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aaa150 msgr2=0x561dc4aabf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aaa150 0x561dc4aabf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 -- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 msgr2=0x561dc4aac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:15.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4aac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef548ef640 1 -- 172.21.15.49:0/550245968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc47eb7d0 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.301+0000 7fef550f0640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4aac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef548ef640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fef440059f0 tx=0x7fef4401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319959fc4f60eeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44021040 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fef4401c410 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef44031070 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef44005110 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 10 client.16185 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561dc4b28920 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44031070 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef4402b160 con 0x561dc4aa9830 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 10 client.16185 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:15.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 20 client.16185 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4b28e30 0x561dc4b2b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 4 client.16185 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.302+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dc4a14d20 con 0x561dc4b28e30 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef4ffff640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4b28e30 0x561dc4b2b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef2ffff640 10 client.16185 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef4ffff640 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4b28e30 0x561dc4b2b220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e0a94b9393f209 server_cookie=1509d6d49dc72e28 in_seq=0 out_seq=0 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef2ffff640 1 -- 172.21.15.49:0/550245968 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dc4a14d20 con 0x561dc4b28e30 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef2ffff640 10 client.16185 handle_command_reply: tid=0 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 client.16185 shutdown 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 msgr2=0x561dc4aad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fef440059f0 tx=0x7fef4401c810 comp rx=0 tx=0).stop 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 shutdown_connections 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561dc4b28e30 0x561dc4b2b220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dc4ab2d30 0x561dc4aac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561dc4aaa150 0x561dc4aabf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 --2- 172.21.15.49:0/550245968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561dc4aa9830 0x561dc4aad530 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 >> 172.21.15.49:0/550245968 conn(0x561dc4aa3f60 msgr2=0x561dc4aa2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef2ffff640 0 client.16185 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 shutdown_connections 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.303+0000 7fef5981c740 1 -- 172.21.15.49:0/550245968 wait complete. 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef5981c740 20 client.16185 trim_cache size 0 max 16384 2024-03-16T08:19:15.310 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 msgr2=0x7fef400a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 0x7fef400a4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x561dc4aa50f0 tx=0x7fef48009b00 comp rx=0 tx=0).stop 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 msgr2=0x7fef501985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009230 tx=0x7fef3c02cb30 comp rx=0 tx=0).stop 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 shutdown_connections 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fef400a2b10 0x7fef400a4fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef50106660 0x7fef5019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef50101010 0x7fef501985e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 --2- 172.21.15.49:0/381390482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef50100650 0x7fef501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:15.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.304+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 >> 172.21.15.49:0/381390482 conn(0x7fef500777e0 msgr2=0x7fef50068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:15.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.305+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 shutdown_connections 2024-03-16T08:19:15.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:15.305+0000 7fef56b53640 1 -- 172.21.15.49:0/381390482 wait complete. 2024-03-16T08:19:15.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:14 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2181111438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:16.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:15 smithi154 ceph-mon[36471]: pgmap v64: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 936 op/s 2024-03-16T08:19:16.261 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:15 smithi129 ceph-mon[36561]: pgmap v64: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 936 op/s 2024-03-16T08:19:16.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:15 smithi049 ceph-mon[31942]: pgmap v64: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 936 op/s 2024-03-16T08:19:16.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:19:16.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:19:16.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 -- 172.21.15.49:0/2150314612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24106b10 msgr2=0x7f9b2410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/2150314612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24106b10 0x7f9b2410e720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009ca0 tx=0x7f9b1c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 -- 172.21.15.49:0/2150314612 shutdown_connections 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/2150314612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24106b10 0x7f9b2410e720 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/2150314612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100d00 0x7f9b24106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/2150314612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24100350 0x7f9b24100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 -- 172.21.15.49:0/2150314612 >> 172.21.15.49:0/2150314612 conn(0x7f9b24075940 msgr2=0x7f9b24075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 -- 172.21.15.49:0/2150314612 shutdown_connections 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.443+0000 7f9b2bcea640 1 -- 172.21.15.49:0/2150314612 wait complete. 2024-03-16T08:19:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 Processor -- start 2024-03-16T08:19:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 -- start start 2024-03-16T08:19:16.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100350 0x7f9b241012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24106b10 0x7f9b24101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24111c00 con 0x7f9b24101d40 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24111a80 con 0x7f9b24106b10 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2bcea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24111d80 con 0x7f9b24100350 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b29285640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24106b10 0x7f9b24101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55380/0 (socket says 172.21.15.49:55380) 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100350 0x7f9b241012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2a287640 1 -- 172.21.15.49:0/4196466298 learned_addr learned my addr 172.21.15.49:0/4196466298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2a287640 1 -- 172.21.15.49:0/4196466298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100350 msgr2=0x7f9b241012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.444+0000 7f9b2a287640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100350 0x7f9b241012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2a287640 1 -- 172.21.15.49:0/4196466298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24106b10 msgr2=0x7f9b24101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2a287640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24106b10 0x7f9b24101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2a287640 1 -- 172.21.15.49:0/4196466298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241a69b0 con 0x7f9b24101d40 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2a287640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c02c700 tx=0x7f9b1c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b1affd640 1 -- 172.21.15.49:0/4196466298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c041020 con 0x7f9b24101d40 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b1affd640 1 -- 172.21.15.49:0/4196466298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c034cd0 con 0x7f9b24101d40 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241a6c40 con 0x7f9b24101d40 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b1affd640 1 -- 172.21.15.49:0/4196466298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c039040 con 0x7f9b24101d40 2024-03-16T08:19:16.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.445+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a7130 con 0x7f9b24101d40 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.447+0000 7f9b1affd640 1 -- 172.21.15.49:0/4196466298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9b1c044020 con 0x7f9b24101d40 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 Processor -- start 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 -- start start 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b1affd640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 0x7f9b100a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d7a30 0x555cf95d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 0x555cf95e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555cf953b0d0 con 0x555cf95d7a30 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555cf94e3c70 con 0x555cf95d8710 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555cf9537170 con 0x555cf95e0da0 2024-03-16T08:19:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b1affd640 1 -- 172.21.15.49:0/4196466298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9b1c0f0940 con 0x7f9b24101d40 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.448+0000 7f9b29285640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 0x7f9b100a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 0x555cf95e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d7a30 0x555cf95d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39924/0 (socket says 172.21.15.49:39924) 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 -- 172.21.15.49:0/1416610255 learned_addr learned my addr 172.21.15.49:0/1416610255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 -- 172.21.15.49:0/1416610255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 msgr2=0x555cf95e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 0x555cf95e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 -- 172.21.15.49:0/1416610255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d7a30 msgr2=0x555cf95d7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d7a30 0x555cf95d7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 -- 172.21.15.49:0/1416610255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf93177e0 con 0x555cf95d8710 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b2a287640 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 0x555cf95e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29285640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 0x7f9b100a4e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b24102d50 tx=0x7f9b14009ae0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009cf0 tx=0x7f9b20011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbd17d48fde68f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.449+0000 7f9b0ffff640 1 -- 172.21.15.49:0/1416610255 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20004030 con 0x555cf95d8710 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b0ffff640 1 -- 172.21.15.49:0/1416610255 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b200041d0 con 0x555cf95d8710 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b0ffff640 1 -- 172.21.15.49:0/1416610255 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200197a0 con 0x555cf95d8710 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1416610255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 msgr2=0x555cf95d7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009cf0 tx=0x7f9b20011810 comp rx=0 tx=0).stop 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1416610255 shutdown_connections 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95e0da0 0x555cf95e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d7a30 0x555cf95d7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1416610255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d8710 0x555cf95d7460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.450+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1416610255 >> 172.21.15.49:0/1416610255 conn(0x555cf959ec50 msgr2=0x555cf95d2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1416610255 shutdown_connections 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1416610255 wait complete. 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- start start 2024-03-16T08:19:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 Processor -- start 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555cf95e3c10 con 0x555cf95d8710 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555cf94f70d0 con 0x555cf95e0da0 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555cf95e3d90 con 0x555cf95d7a30 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.451+0000 7f9b2e9b3740 10 client.0 _ll_register_callbacks cb 0x555cf94c0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37824/0 (socket says 172.21.15.49:37824) 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29a86640 1 -- 172.21.15.49:0/1380515312 learned_addr learned my addr 172.21.15.49:0/1380515312 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b2a287640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 -- 172.21.15.49:0/1380515312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 msgr2=0x555cf95ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 -- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 msgr2=0x555cf95de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95de720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 -- 172.21.15.49:0/1380515312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf9377f70 con 0x555cf95d8710 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b2a287640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b29285640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95de1e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9b24073ed0 tx=0x7f9b1401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c1f22be91f6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14024070 con 0x555cf95d8710 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9b14005020 con 0x555cf95d8710 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b200041d0 con 0x555cf95d8710 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9b14005220 con 0x555cf95d8710 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.452+0000 7f9b2e9b3740 10 client.16197 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555cf920f170 con 0x555cf95d8710 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b200041d0 con 0x555cf95d8710 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9b1402a460 con 0x555cf95d8710 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 10 client.16197 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 20 client.16197 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf95f0db0 0x555cf95f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 4 client.16197 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf9548610 con 0x555cf95f0db0 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.453+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf95f0db0 0x555cf95f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.454+0000 7f9b29a86640 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf95f0db0 0x555cf95f31d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38365b56107425d1 server_cookie=f3d6c81f6481c3dc in_seq=0 out_seq=0 2024-03-16T08:19:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.454+0000 7f9b0d7fa640 10 client.16197 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.454+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/1380515312 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555cf9548610 con 0x555cf95f0db0 2024-03-16T08:19:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b0d7fa640 10 client.16197 handle_command_reply: tid=0 2024-03-16T08:19:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 client.16197 shutdown 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 msgr2=0x555cf95de1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95de1e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9b24073ed0 tx=0x7f9b1401d000 comp rx=0 tx=0).stop 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 shutdown_connections 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf95f0db0 0x555cf95f31d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95de1e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b0d7fa640 0 client.16197 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/1380515312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 >> 172.21.15.49:0/1380515312 conn(0x555cf959ec50 msgr2=0x555cf95d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 shutdown_connections 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/1380515312 wait complete. 2024-03-16T08:19:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.455+0000 7f9b2e9b3740 20 client.16197 trim_cache size 0 max 16384 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 Processor -- start 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 -- start start 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d7a30 0x555cf95e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d8710 0x555cf95eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241b1ee0 con 0x555cf95e0da0 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241af7b0 con 0x555cf95d7a30 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241ad620 con 0x555cf95d8710 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b29285640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d8710 0x555cf95eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d7a30 0x555cf95e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55406/0 (socket says 172.21.15.49:55406) 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 -- 172.21.15.49:0/677751601 learned_addr learned my addr 172.21.15.49:0/677751601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 -- 172.21.15.49:0/677751601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d8710 msgr2=0x555cf95eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d8710 0x555cf95eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 -- 172.21.15.49:0/677751601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d7a30 msgr2=0x555cf95e03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d7a30 0x555cf95e03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.475+0000 7f9b2a287640 1 -- 172.21.15.49:0/677751601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf95d6a60 con 0x555cf95e0da0 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.476+0000 7f9b2a287640 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0b9ad0 tx=0x7f9b1c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c2e7b7ceec9cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.476+0000 7f9b0f7fe640 1 -- 172.21.15.49:0/677751601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c034920 con 0x555cf95e0da0 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.476+0000 7f9b0f7fe640 1 -- 172.21.15.49:0/677751601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c0098e0 con 0x555cf95e0da0 2024-03-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.476+0000 7f9b0f7fe640 1 -- 172.21.15.49:0/677751601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0bd100 con 0x555cf95e0da0 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/677751601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 msgr2=0x555cf95f8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0b9ad0 tx=0x7f9b1c003c30 comp rx=0 tx=0).stop 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/677751601 shutdown_connections 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95e0da0 0x555cf95f8a50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d8710 0x555cf95eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/677751601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95d7a30 0x555cf95e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/677751601 >> 172.21.15.49:0/677751601 conn(0x555cf959ec50 msgr2=0x555cf95de4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/677751601 shutdown_connections 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/677751601 wait complete. 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 -- start start 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.477+0000 7f9b2e9b3740 1 Processor -- start 2024-03-16T08:19:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241adc70 con 0x555cf95d8710 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241af0e0 con 0x555cf95e0da0 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241ae340 con 0x555cf95d7a30 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2e9b3740 10 client.0 _ll_register_callbacks cb 0x7f9b14009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29a86640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55418/0 (socket says 172.21.15.49:55418) 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 -- 172.21.15.49:0/2556516370 learned_addr learned my addr 172.21.15.49:0/2556516370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2a287640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 -- 172.21.15.49:0/2556516370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 msgr2=0x555cf95e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95e9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 -- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 msgr2=0x555cf95e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 -- 172.21.15.49:0/2556516370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf95e6bd0 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29a86640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b2a287640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b29285640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x555cf95df270 tx=0x7f9b14005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e17e4250ca89ade server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.478+0000 7f9b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14024040 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9b14009d30 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b1c0098e0 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9b1400fca0 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 10 client.16203 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555cf9657f00 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c0098e0 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9b14004c80 con 0x555cf95d8710 2024-03-16T08:19:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 10 client.16203 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 20 client.16203 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf9658650 0x555cf965aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 4 client.16203 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.479+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555cf9542960 con 0x555cf9658650 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.480+0000 7f9b29a86640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf9658650 0x555cf965aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.480+0000 7f9b0d7fa640 10 client.16203 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.480+0000 7f9b29a86640 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf9658650 0x555cf965aa70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c33216430e02e5 server_cookie=c121e5e5ac743b6f in_seq=0 out_seq=0 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b0d7fa640 1 -- 172.21.15.49:0/2556516370 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555cf9542960 con 0x555cf9658650 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b0d7fa640 10 client.16203 handle_command_reply: tid=0 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 client.16203 shutdown 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 msgr2=0x555cf95e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x555cf95df270 tx=0x7f9b14005670 comp rx=0 tx=0).stop 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 shutdown_connections 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555cf9658650 0x555cf965aa70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cf95e0da0 0x555cf95e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf95d8710 0x555cf95e6050 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 --2- 172.21.15.49:0/2556516370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555cf95d7a30 0x555cf95e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b0d7fa640 0 client.16203 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 >> 172.21.15.49:0/2556516370 conn(0x555cf959ec50 msgr2=0x555cf95d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 shutdown_connections 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 1 -- 172.21.15.49:0/2556516370 wait complete. 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.481+0000 7f9b2e9b3740 20 client.16203 trim_cache size 0 max 16384 2024-03-16T08:19:16.487 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:16.488 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:19:16.488 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "ad25d2bb-8850-453e-a2d4-b1e98f3304f7", 2024-03-16T08:19:16.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:19:16.488 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 msgr2=0x7f9b100a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 0x7f9b100a4e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b24102d50 tx=0x7f9b14009ae0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 msgr2=0x7f9b241a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c02c700 tx=0x7f9b1c032f20 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 shutdown_connections 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9b100a29a0 0x7f9b100a4e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b24101d40 0x7f9b241a62b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b24106b10 0x7f9b24101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 --2- 172.21.15.49:0/4196466298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b24100350 0x7f9b241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 >> 172.21.15.49:0/4196466298 conn(0x7f9b24075940 msgr2=0x7f9b240fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 shutdown_connections 2024-03-16T08:19:16.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.482+0000 7f9b2bcea640 1 -- 172.21.15.49:0/4196466298 wait complete. 2024-03-16T08:19:16.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:19:16.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:19:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 -- 172.21.15.49:0/3442028928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c107790 msgr2=0x7fbb4c107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 --2- 172.21.15.49:0/3442028928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c107790 0x7fbb4c107b70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 -- 172.21.15.49:0/3442028928 shutdown_connections 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 --2- 172.21.15.49:0/3442028928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 --2- 172.21.15.49:0/3442028928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c069d50 0x7fbb4c06a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 --2- 172.21.15.49:0/3442028928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c107790 0x7fbb4c107b70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 -- 172.21.15.49:0/3442028928 >> 172.21.15.49:0/3442028928 conn(0x7fbb4c0775d0 msgr2=0x7fbb4c0779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 -- 172.21.15.49:0/3442028928 shutdown_connections 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.617+0000 7fbb520e8640 1 -- 172.21.15.49:0/3442028928 wait complete. 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 Processor -- start 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 -- start start 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c107790 0x7fbb4c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c111b80 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c111a00 con 0x7fbb4c107790 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb520e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c111d00 con 0x7fbb4c069d50 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55434/0 (socket says 172.21.15.49:55434) 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37870/0 (socket says 172.21.15.49:37870) 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4affd640 1 -- 172.21.15.49:0/2583188132 learned_addr learned my addr 172.21.15.49:0/2583188132 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.618+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c107790 0x7fbb4c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 -- 172.21.15.49:0/2583188132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 msgr2=0x7fbb4c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 -- 172.21.15.49:0/2583188132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c107790 msgr2=0x7fbb4c19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c107790 0x7fbb4c19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 -- 172.21.15.49:0/2583188132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c19d070 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb4affd640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000e9f0 tx=0x7fbb4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb48ff9640 1 -- 172.21.15.49:0/2583188132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000cd80 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c19d360 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb48ff9640 1 -- 172.21.15.49:0/2583188132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb40010910 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb48ff9640 1 -- 172.21.15.49:0/2583188132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb400188f0 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.619+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c1a4bf0 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.622+0000 7fbb48ff9640 1 -- 172.21.15.49:0/2583188132 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fbb40010ac0 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.622+0000 7fbb54dac740 1 Processor -- start 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.622+0000 7fbb54dac740 1 -- start start 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665ca4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665cafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557665c08500 con 0x557665cad650 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557665bb0be0 con 0x557665ca4150 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb54dac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557665c045a0 con 0x557665ca4a70 2024-03-16T08:19:16.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665cafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665ca4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37884/0 (socket says 172.21.15.49:37884) 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/4213541825 learned_addr learned my addr 172.21.15.49:0/4213541825 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665ca4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:39976/0 (socket says 172.21.15.49:39976) 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb48ff9640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 0x7fbb340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/4213541825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 msgr2=0x557665ca4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665ca4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/4213541825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 msgr2=0x557665cafa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665cafa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/4213541825 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557665bc44b0 con 0x557665ca4a70 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 0x7fbb340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb48ff9640 1 -- 172.21.15.49:0/2583188132 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbb400cf1c0 con 0x7fbb4c06a6f0 2024-03-16T08:19:16.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.623+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c02c170 tx=0x7fbb3c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a91cd162a39b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb31ffb640 1 -- 172.21.15.49:0/4213541825 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c042070 con 0x557665ca4a70 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 0x7fbb340a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb44005c40 tx=0x7fbb44004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb31ffb640 1 -- 172.21.15.49:0/4213541825 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c004c70 con 0x557665ca4a70 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb31ffb640 1 -- 172.21.15.49:0/4213541825 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005260 con 0x557665ca4a70 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 -- 172.21.15.49:0/4213541825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 msgr2=0x557665cad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c02c170 tx=0x7fbb3c009560 comp rx=0 tx=0).stop 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 -- 172.21.15.49:0/4213541825 shutdown_connections 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665cafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4a70 0x557665cad110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 --2- 172.21.15.49:0/4213541825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665ca4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.624+0000 7fbb54dac740 1 -- 172.21.15.49:0/4213541825 >> 172.21.15.49:0/4213541825 conn(0x557665c9e880 msgr2=0x557665c9ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.625+0000 7fbb54dac740 1 -- 172.21.15.49:0/4213541825 shutdown_connections 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.625+0000 7fbb54dac740 1 -- 172.21.15.49:0/4213541825 wait complete. 2024-03-16T08:19:16.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.625+0000 7fbb54dac740 1 -- start start 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.625+0000 7fbb54dac740 1 Processor -- start 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.625+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665caa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 0x557665cab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557665cb03e0 con 0x557665ca4a70 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557665bc4160 con 0x557665cad650 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557665cb0560 con 0x557665ca4150 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 10 client.0 _ll_register_callbacks cb 0x557665ba8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55454/0 (socket says 172.21.15.49:55454) 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 -- 172.21.15.49:0/3358110121 learned_addr learned my addr 172.21.15.49:0/3358110121 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 0x557665cab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665caa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 -- 172.21.15.49:0/3358110121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 msgr2=0x557665caa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665caa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 -- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 msgr2=0x557665cab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 0x557665cab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 -- 172.21.15.49:0/3358110121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576659e67d0 con 0x557665ca4a70 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 0x557665cab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb4affd640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbb400982f0 tx=0x7fbb4009b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b928140987d9725c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb2affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40084e60 con 0x557665ca4a70 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbb400862b0 con 0x557665ca4a70 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.626+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557665c15a40 con 0x557665ca4a70 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbb400ccdf0 con 0x557665ca4a70 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 10 client.16215 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55766591c3f0 con 0x557665ca4a70 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557665c15a40 con 0x557665ca4a70 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbb4009ae90 con 0x557665ca4a70 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 10 client.16215 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 20 client.16215 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665cbd690 0x557665cbfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 4 client.16215 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557665c0fd90 con 0x557665cbd690 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.627+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665cbd690 0x557665cbfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.628+0000 7fbb2affd640 10 client.16215 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.628+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665cbd690 0x557665cbfa80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f3939c5af32294 server_cookie=d8c581743a597805 in_seq=0 out_seq=0 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.628+0000 7fbb2affd640 1 -- 172.21.15.49:0/3358110121 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557665c0fd90 con 0x557665cbd690 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.628+0000 7fbb2affd640 10 client.16215 handle_command_reply: tid=0 2024-03-16T08:19:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.628+0000 7fbb54dac740 1 client.16215 shutdown 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 msgr2=0x557665caac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbb400982f0 tx=0x7fbb4009b810 comp rx=0 tx=0).stop 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 shutdown_connections 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665cbd690 0x557665cbfa80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665cad650 0x557665cab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665caac30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 --2- 172.21.15.49:0/3358110121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665caa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 >> 172.21.15.49:0/3358110121 conn(0x557665c9e880 msgr2=0x557665ca05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb2affd640 0 client.16215 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 shutdown_connections 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 1 -- 172.21.15.49:0/3358110121 wait complete. 2024-03-16T08:19:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.629+0000 7fbb54dac740 20 client.16215 trim_cache size 0 max 16384 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 Processor -- start 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 -- start start 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665cace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 0x557665cb9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1afb60 con 0x557665cad650 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ad430 con 0x557665ca4a70 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb54dac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ab280 con 0x557665ca4150 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665cace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 0x557665cb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55458/0 (socket says 172.21.15.49:55458) 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4bfff640 1 -- 172.21.15.49:0/2454052679 learned_addr learned my addr 172.21.15.49:0/2454052679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/2454052679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 msgr2=0x557665cb9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 0x557665cb9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/2454052679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 msgr2=0x557665c9d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/2454052679 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576658dc0f0 con 0x557665ca4150 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4affd640 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 0x557665cb9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.649+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.650+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665cace20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c036040 tx=0x7fbb3c004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49315890e8971e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.650+0000 7fbb317fa640 1 -- 172.21.15.49:0/2454052679 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c042040 con 0x557665ca4150 2024-03-16T08:19:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.650+0000 7fbb317fa640 1 -- 172.21.15.49:0/2454052679 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c050b90 con 0x557665ca4150 2024-03-16T08:19:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.650+0000 7fbb317fa640 1 -- 172.21.15.49:0/2454052679 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c034440 con 0x557665ca4150 2024-03-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- 172.21.15.49:0/2454052679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 msgr2=0x557665cace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665cace20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c036040 tx=0x7fbb3c004f20 comp rx=0 tx=0).stop 2024-03-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- 172.21.15.49:0/2454052679 shutdown_connections 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665cad650 0x557665c9d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4a70 0x557665cb9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 --2- 172.21.15.49:0/2454052679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665ca4150 0x557665cace20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- 172.21.15.49:0/2454052679 >> 172.21.15.49:0/2454052679 conn(0x557665c9e880 msgr2=0x557665c9d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- 172.21.15.49:0/2454052679 shutdown_connections 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- 172.21.15.49:0/2454052679 wait complete. 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 -- start start 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.651+0000 7fbb54dac740 1 Processor -- start 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665cb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665cad650 0x557665cb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ab8f0 con 0x557665ca4a70 2024-03-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1acd60 con 0x557665ca4150 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1abfc0 con 0x557665cad650 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb54dac740 10 client.0 _ll_register_callbacks cb 0x7fbb400d29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40020/0 (socket says 172.21.15.49:40020) 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/1858216787 learned_addr learned my addr 172.21.15.49:0/1858216787 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4bfff640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665cad650 0x557665cb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4affd640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665cb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/1858216787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665cad650 msgr2=0x557665cb1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665cad650 0x557665cb1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/1858216787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 msgr2=0x557665cb1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665cb1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 -- 172.21.15.49:0/1858216787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557665bcf9d0 con 0x557665ca4150 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb4b7fe640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005ec0 tx=0x7fbb3c005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b73179f745d2911 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.652+0000 7fbb2affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c042040 con 0x557665ca4150 2024-03-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3c050b90 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbb3c005420 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbb3c05d020 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 10 client.26102 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557665d23990 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c050b90 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbb3c0505b0 con 0x557665ca4150 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 10 client.26102 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 20 client.26102 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665d23c20 0x557665d26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 4 client.26102 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.653+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557665b8d540 con 0x557665d23c20 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb4affd640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665d23c20 0x557665d26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb4affd640 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665d23c20 0x557665d26010 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915ef34c1bf498a0 server_cookie=340e12303f60cc37 in_seq=0 out_seq=0 2024-03-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb2affd640 10 client.26102 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb2affd640 1 -- 172.21.15.49:0/1858216787 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557665b8d540 con 0x557665d23c20 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb2affd640 10 client.26102 handle_command_reply: tid=0 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 client.26102 shutdown 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 msgr2=0x557665cb1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005ec0 tx=0x7fbb3c005320 comp rx=0 tx=0).stop 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 shutdown_connections 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x557665d23c20 0x557665d26010 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557665cad650 0x557665cb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557665ca4a70 0x557665cb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 --2- 172.21.15.49:0/1858216787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557665ca4150 0x557665cb1050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb2affd640 0 client.26102 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 >> 172.21.15.49:0/1858216787 conn(0x557665c9e880 msgr2=0x557665caaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 shutdown_connections 2024-03-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.654+0000 7fbb54dac740 1 -- 172.21.15.49:0/1858216787 wait complete. 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb54dac740 20 client.26102 trim_cache size 0 max 16384 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "ad25d2bb-8850-453e-a2d4-b1e98f3304f7": { 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "ad25d2bb-8850-453e-a2d4-b1e98f3304f7", 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 msgr2=0x7fbb340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 0x7fbb340a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb44005c40 tx=0x7fbb44004a40 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 msgr2=0x7fbb4c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000e9f0 tx=0x7fbb4000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 shutdown_connections 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbb340a2970 0x7fbb340a4e30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb4c107790 0x7fbb4c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c06a6f0 0x7fbb4c1985c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 --2- 172.21.15.49:0/2583188132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb4c069d50 0x7fbb4c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 >> 172.21.15.49:0/2583188132 conn(0x7fbb4c0775d0 msgr2=0x7fbb4c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 shutdown_connections 2024-03-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:16.655+0000 7fbb520e8640 1 -- 172.21.15.49:0/2583188132 wait complete. 2024-03-16T08:19:16.668 INFO:tasks.cephfs.filesystem:scrub status for tag:ad25d2bb-8850-453e-a2d4-b1e98f3304f7 - {'path': '/', 'tag': 'ad25d2bb-8850-453e-a2d4-b1e98f3304f7', 'options': 'recursive,force'} 2024-03-16T08:19:18.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:19:18.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:19:18.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: pgmap v65: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 670 op/s 2024-03-16T08:19:18.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:17 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:19:18.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:19:18.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:19:18.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: pgmap v65: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 670 op/s 2024-03-16T08:19:18.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:17 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:19:18.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:19:18.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:19:18.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:18.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: pgmap v65: 97 pgs: 97 active+clean; 259 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 670 op/s 2024-03-16T08:19:18.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:17 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:19:20.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:19 smithi154 ceph-mon[36471]: pgmap v66: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 2.65k op/s 2024-03-16T08:19:20.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:19 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:19:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:19 smithi049 ceph-mon[31942]: pgmap v66: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 2.65k op/s 2024-03-16T08:19:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:19 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:19:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:19 smithi129 ceph-mon[36561]: pgmap v66: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 2.65k op/s 2024-03-16T08:19:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:19 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:19:21.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:19:21.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:19:21.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:20 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:19:21.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:19:21.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:20 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:19:21.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:20 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:19:22.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:21 smithi049 ceph-mon[31942]: pgmap v67: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:21 smithi129 ceph-mon[36561]: pgmap v67: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:21 smithi154 ceph-mon[36471]: pgmap v67: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:23 smithi049 ceph-mon[31942]: pgmap v68: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:23 smithi129 ceph-mon[36561]: pgmap v68: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:23 smithi154 ceph-mon[36471]: pgmap v68: 97 pgs: 97 active+clean; 313 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 2.31k op/s 2024-03-16T08:19:26.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:25 smithi049 ceph-mon[31942]: pgmap v69: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 2.32k op/s 2024-03-16T08:19:26.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:25 smithi129 ceph-mon[36561]: pgmap v69: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 2.32k op/s 2024-03-16T08:19:26.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:25 smithi154 ceph-mon[36471]: pgmap v69: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 2.32k op/s 2024-03-16T08:19:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- 172.21.15.49:0/2981013357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 msgr2=0x7fcdd8105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 --2- 172.21.15.49:0/2981013357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8105a60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009900 tx=0x7fcdc802bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- 172.21.15.49:0/2981013357 shutdown_connections 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 --2- 172.21.15.49:0/2981013357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd81012f0 0x7fcdd810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 --2- 172.21.15.49:0/2981013357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd8100930 0x7fcdd8100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 --2- 172.21.15.49:0/2981013357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8105a60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- 172.21.15.49:0/2981013357 >> 172.21.15.49:0/2981013357 conn(0x7fcdd8077530 msgr2=0x7fcdd8077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- 172.21.15.49:0/2981013357 shutdown_connections 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- 172.21.15.49:0/2981013357 wait complete. 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 Processor -- start 2024-03-16T08:19:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.507+0000 7fcdde80b640 1 -- start start 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd8100930 0x7fcdd8102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 0x7fcdd8102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8110ba0 con 0x7fcdd8105680 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8110a20 con 0x7fcdd81012f0 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdde80b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8110d20 con 0x7fcdd8100930 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 0x7fcdd8102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40150/0 (socket says 172.21.15.49:40150) 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 -- 172.21.15.49:0/1229374198 learned_addr learned my addr 172.21.15.49:0/1229374198 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd8100930 0x7fcdd8102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 -- 172.21.15.49:0/1229374198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd8100930 msgr2=0x7fcdd8102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd8100930 0x7fcdd8102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 -- 172.21.15.49:0/1229374198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 msgr2=0x7fcdd8102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 0x7fcdd8102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcddcda8640 1 -- 172.21.15.49:0/1229374198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd81a6520 con 0x7fcdd8105680 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.508+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 0x7fcdd8102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcddcda8640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ea60 tx=0x7fcdd000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcdd57fa640 1 -- 172.21.15.49:0/1229374198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000ce40 con 0x7fcdd8105680 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd81a6810 con 0x7fcdd8105680 2024-03-16T08:19:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcdd57fa640 1 -- 172.21.15.49:0/1229374198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd0005020 con 0x7fcdd8105680 2024-03-16T08:19:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcdd57fa640 1 -- 172.21.15.49:0/1229374198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0010690 con 0x7fcdd8105680 2024-03-16T08:19:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.509+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd81a6d60 con 0x7fcdd8105680 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.511+0000 7fcdd57fa640 1 -- 172.21.15.49:0/1229374198 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fcdd00051d0 con 0x7fcdd8105680 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 Processor -- start 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 -- start start 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd4672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd467820 0x5577cd469c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5577cd3c2a40 con 0x5577cd45ea50 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5577cd36b920 con 0x5577cd467820 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcde13f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577cd3beae0 con 0x5577cd45e130 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd57fa640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 0x7fcdc00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd4672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:53434/0 (socket says 172.21.15.49:53434) 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2169130101 learned_addr learned my addr 172.21.15.49:0/2169130101 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcddcda8640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 0x7fcdc00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd467820 0x5577cd469c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.512+0000 7fcdd57fa640 1 -- 172.21.15.49:0/1229374198 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcdd00cf3b0 con 0x7fcdd8105680 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2169130101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd467820 msgr2=0x5577cd469c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd467820 0x5577cd469c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2169130101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 msgr2=0x5577cd4672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd4672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2169130101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cd383460 con 0x5577cd45e130 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009fd0 tx=0x7fcdcc009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5fb46f0102cc25 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcddcda8640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 0x7fcdc00a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcdd009b760 tx=0x7fcdd0086040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdbe7fc640 1 -- 172.21.15.49:0/2169130101 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00fc50 con 0x5577cd45e130 2024-03-16T08:19:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdbe7fc640 1 -- 172.21.15.49:0/2169130101 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc00fdf0 con 0x5577cd45e130 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.513+0000 7fcdbe7fc640 1 -- 172.21.15.49:0/2169130101 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00e6a0 con 0x5577cd45e130 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- 172.21.15.49:0/2169130101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 msgr2=0x5577cd45e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009fd0 tx=0x7fcdcc009ba0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- 172.21.15.49:0/2169130101 shutdown_connections 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd467820 0x5577cd469c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd4672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2169130101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45e130 0x5577cd45e510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- 172.21.15.49:0/2169130101 >> 172.21.15.49:0/2169130101 conn(0x5577cd458a50 msgr2=0x5577cd458e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- 172.21.15.49:0/2169130101 shutdown_connections 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- 172.21.15.49:0/2169130101 wait complete. 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 -- start start 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.514+0000 7fcde13f9740 1 Processor -- start 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd4653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5577cd46a450 con 0x5577cd45ea50 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5577cd37ed70 con 0x5577cd45e130 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577cd46a5d0 con 0x5577cd467820 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcde13f9740 10 client.0 _ll_register_callbacks cb 0x5577cd0d6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:54260/0 (socket says 172.21.15.49:54260) 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40168/0 (socket says 172.21.15.49:40168) 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2824838726 learned_addr learned my addr 172.21.15.49:0/2824838726 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd4653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2824838726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 msgr2=0x5577cd4653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd4653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2824838726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 msgr2=0x5577cd464e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd77fe640 1 -- 172.21.15.49:0/2824838726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cd0d7210 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.515+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006010 tx=0x7fcdcc0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc08dbed831d7cee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0046c0 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcdcc028040 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc00fdf0 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc004860 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 10 client.26114 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5577cd1a3620 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc00fdf0 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcdcc01d020 con 0x5577cd45e130 2024-03-16T08:19:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 10 client.26114 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 20 client.26114 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4778e0 0x5577cd479cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 4 client.26114 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.516+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577cd3cff80 con 0x5577cd4778e0 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.517+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4778e0 0x5577cd479cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.517+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4778e0 0x5577cd479cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc195a6498a27dd server_cookie=cde6e038a97e65d in_seq=0 out_seq=0 2024-03-16T08:19:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.517+0000 7fcdb77fe640 10 client.26114 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcdb77fe640 1 -- 172.21.15.49:0/2824838726 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5577cd3cff80 con 0x5577cd4778e0 2024-03-16T08:19:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcdb77fe640 10 client.26114 handle_command_reply: tid=0 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 client.26114 shutdown 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 msgr2=0x5577cd464920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006010 tx=0x7fcdcc0096c0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 shutdown_connections 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4778e0 0x5577cd479cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd4653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcdb77fe640 0 client.26114 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd464e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 --2- 172.21.15.49:0/2824838726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd464920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 >> 172.21.15.49:0/2824838726 conn(0x5577cd458a50 msgr2=0x5577cd45a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 shutdown_connections 2024-03-16T08:19:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.518+0000 7fcde13f9740 1 -- 172.21.15.49:0/2824838726 wait complete. 2024-03-16T08:19:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.519+0000 7fcde13f9740 20 client.26114 trim_cache size 0 max 16384 2024-03-16T08:19:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 Processor -- start 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 -- start start 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd467820 0x5577cd4648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b1e40 con 0x5577cd467820 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81af710 con 0x5577cd45e130 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcde13f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81ad560 con 0x5577cd45ea50 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:53446/0 (socket says 172.21.15.49:53446) 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:54276/0 (socket says 172.21.15.49:54276) 2024-03-16T08:19:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1702363535 learned_addr learned my addr 172.21.15.49:0/1702363535 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.551+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd467820 0x5577cd4648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1702363535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 msgr2=0x5577cd4731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1702363535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd467820 msgr2=0x5577cd4648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd467820 0x5577cd4648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1702363535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cd469f80 con 0x5577cd45ea50 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009230 tx=0x7fcdc8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69f59db550e1477 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdbdffb640 1 -- 172.21.15.49:0/1702363535 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8050070 con 0x5577cd45ea50 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdbdffb640 1 -- 172.21.15.49:0/1702363535 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc80046c0 con 0x5577cd45ea50 2024-03-16T08:19:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.552+0000 7fcdbdffb640 1 -- 172.21.15.49:0/1702363535 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8005220 con 0x5577cd45ea50 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- 172.21.15.49:0/1702363535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 msgr2=0x5577cd47f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009230 tx=0x7fcdc8005fd0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- 172.21.15.49:0/1702363535 shutdown_connections 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd467820 0x5577cd4648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd45ea50 0x5577cd47f4a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1702363535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- 172.21.15.49:0/1702363535 >> 172.21.15.49:0/1702363535 conn(0x5577cd458a50 msgr2=0x5577cd45fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- 172.21.15.49:0/1702363535 shutdown_connections 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- 172.21.15.49:0/1702363535 wait complete. 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 -- start start 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.553+0000 7fcde13f9740 1 Processor -- start 2024-03-16T08:19:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd462150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81adbd0 con 0x5577cd45ea50 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81af040 con 0x5577cd45e130 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81ae2a0 con 0x5577cd467820 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcde13f9740 10 client.0 _ll_register_callbacks cb 0x7fcdd00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40190/0 (socket says 172.21.15.49:40190) 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1671335451 learned_addr learned my addr 172.21.15.49:0/1671335451 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd462150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1671335451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 msgr2=0x5577cd462150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd462150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 msgr2=0x5577cd4616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 -- 172.21.15.49:0/1671335451 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cd47a270 con 0x5577cd45ea50 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd462150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdd7fff640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcdc802c170 tx=0x7fcdc8004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b5efc5224acb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.554+0000 7fcdb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8050040 con 0x5577cd45ea50 2024-03-16T08:19:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcdc802cab0 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc80046c0 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcdc8004870 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 10 client.16239 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5577cd4dddf0 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc80046c0 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcdc8047070 con 0x5577cd45ea50 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 10 client.16239 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 20 client.16239 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4de4e0 0x5577cd4e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 4 client.16239 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T08:19:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577cd3ca2d0 con 0x5577cd4de4e0 2024-03-16T08:19:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.555+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4de4e0 0x5577cd4e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.556+0000 7fcdb77fe640 10 client.16239 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.556+0000 7fcdd77fe640 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4de4e0 0x5577cd4e0900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086b4c4757289f8 server_cookie=2f37e4123fddd6ef in_seq=0 out_seq=0 2024-03-16T08:19:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.565+0000 7fcdb77fe640 1 -- 172.21.15.49:0/1671335451 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577cd3ca2d0 con 0x5577cd4de4e0 2024-03-16T08:19:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.565+0000 7fcdb77fe640 10 client.16239 handle_command_reply: tid=0 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.565+0000 7fcde13f9740 1 client.16239 shutdown 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 msgr2=0x5577cd461c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcdc802c170 tx=0x7fcdc8004370 comp rx=0 tx=0).stop 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 shutdown_connections 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5577cd4de4e0 0x5577cd4e0900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577cd467820 0x5577cd462150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577cd45ea50 0x5577cd461c10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 --2- 172.21.15.49:0/1671335451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5577cd45e130 0x5577cd4616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcdb77fe640 0 client.16239 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 >> 172.21.15.49:0/1671335451 conn(0x5577cd458a50 msgr2=0x5577cd4648e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 shutdown_connections 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 1 -- 172.21.15.49:0/1671335451 wait complete. 2024-03-16T08:19:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcde13f9740 20 client.16239 trim_cache size 0 max 16384 2024-03-16T08:19:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-16T08:19:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-16T08:19:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-16T08:19:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 msgr2=0x7fcdc00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 0x7fcdc00a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcdd009b760 tx=0x7fcdd0086040 comp rx=0 tx=0).stop 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 msgr2=0x7fcdd8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.566+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ea60 tx=0x7fcdd000ef20 comp rx=0 tx=0).stop 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 shutdown_connections 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcdc00a2a10 0x7fcdc00a4ed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd8105680 0x7fcdd8102dd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd81012f0 0x7fcdd8102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 --2- 172.21.15.49:0/1229374198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdd8100930 0x7fcdd8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 >> 172.21.15.49:0/1229374198 conn(0x7fcdd8077530 msgr2=0x7fcdd80690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 shutdown_connections 2024-03-16T08:19:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-16T08:19:26.567+0000 7fcdde80b640 1 -- 172.21.15.49:0/1229374198 wait complete. 2024-03-16T08:19:28.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:27 smithi049 ceph-mon[31942]: pgmap v70: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 1.98k op/s 2024-03-16T08:19:28.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:27 smithi129 ceph-mon[36561]: pgmap v70: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 1.98k op/s 2024-03-16T08:19:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:27 smithi154 ceph-mon[36471]: pgmap v70: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 1.98k op/s 2024-03-16T08:19:30.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:29 smithi049 ceph-mon[31942]: pgmap v71: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-03-16T08:19:30.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:29 smithi129 ceph-mon[36561]: pgmap v71: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-03-16T08:19:30.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:29 smithi154 ceph-mon[36471]: pgmap v71: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-03-16T08:19:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:31 smithi049 ceph-mon[31942]: pgmap v72: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:31 smithi129 ceph-mon[36561]: pgmap v72: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:32.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:31 smithi154 ceph-mon[36471]: pgmap v72: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:34.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:33 smithi049 ceph-mon[31942]: pgmap v73: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:33 smithi129 ceph-mon[36561]: pgmap v73: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:34.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:33 smithi154 ceph-mon[36471]: pgmap v73: 97 pgs: 97 active+clean; 314 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 10 op/s 2024-03-16T08:19:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:35 smithi049 ceph-mon[31942]: pgmap v74: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-03-16T08:19:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:35 smithi129 ceph-mon[36561]: pgmap v74: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-03-16T08:19:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:35 smithi154 ceph-mon[36471]: pgmap v74: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-03-16T08:19:37.534 INFO:tasks.workunit.client.0.smithi049.stdout:setting up random write file 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:done setting up random write file 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:starting fsync run 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0072s 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0043s 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0042s 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0039s 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0037s 2024-03-16T08:19:37.535 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0046s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0044s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0043s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0043s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0039s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0040s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0038s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0042s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0040s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0044s 2024-03-16T08:19:37.536 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0039s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0040s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0042s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0040s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0039s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0042s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0037s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0037s 2024-03-16T08:19:37.537 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0004s fsync time: 0.0038s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0038s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0038s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0032s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0034s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0036s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0034s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0039s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0039s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0036s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0035s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0035s 2024-03-16T08:19:37.538 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0034s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0773s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0037s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0040s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0046s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0044s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0038s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0044s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0042s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.539 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0038s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0042s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0039s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0004s fsync time: 0.0417s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0038s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0039s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:write time: 0.0002s fsync time: 0.0041s 2024-03-16T08:19:37.540 INFO:tasks.workunit.client.0.smithi049.stdout:run done 61 fsyncs total, killing random writer 2024-03-16T08:19:37.541 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-16T08:19:37.541 INFO:tasks.workunit.client.0.smithi049.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-16T08:19:37.541 INFO:tasks.workunit.client.0.smithi049.stderr:+ whereis lsof 2024-03-16T08:19:37.546 INFO:tasks.workunit.client.0.smithi049.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-03-16T08:19:37.546 INFO:tasks.workunit.client.0.smithi049.stderr:+ lsof 2024-03-16T08:19:37.564 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/a5d0d1b606dce15f3f270d90a04e7172b72a8fca45a7941c0372fd412011390a/merged 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5c08e34925db427309754436642b0e0d25cf80d17c54fa8bd11b2bd41668071c/merged 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/47cccdebf9a954c324c2c3d10d5bc582f813381985d0088edc6930b624b18c14/merged 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0e7f046739cb890bb74f1423846c0c4d4d45f08920a8f780401aa18a8dbc423d/merged 2024-03-16T08:19:37.565 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/9ae365f523b9886c716f6f89a85440f85fbd78f1a27cc8970380e51bee95f7ed/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/14ca838cbae67568c71ad5ca96c99d34f3a0860f82e476fe3de348368574e759/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/627d1a500c24b5643ef2cba9ff077a5555f4cf5910c3626e96a20c35db70388e/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4c2f32176a7a6537d5f3626bb2a801cfa68c1b86f40af57545413a46950afa7f/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/867669d977492925c445659f8cd066f3c5dc015c5221e77e00c75f7743fdeca5/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/dc43cf8e4827267ce30a3b9f3abd2fdd805f129db4e7fd6ac43a41dba7cd6e1c/merged 2024-03-16T08:19:37.566 INFO:tasks.workunit.client.0.smithi049.stderr: Output information may be incomplete. 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-03-16T08:19:37.622 INFO:tasks.workunit.client.0.smithi049.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-03-16T08:19:37.623 INFO:tasks.workunit.client.0.smithi049.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-03-16T08:19:37.624 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-03-16T08:19:37.625 INFO:tasks.workunit.client.0.smithi049.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-03-16T08:19:37.626 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-03-16T08:19:37.627 INFO:tasks.workunit.client.0.smithi049.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-03-16T08:19:37.628 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-03-16T08:19:37.629 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-03-16T08:19:37.630 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-03-16T08:19:37.631 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-03-16T08:19:37.632 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-03-16T08:19:37.633 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-03-16T08:19:37.634 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-03-16T08:19:37.635 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-03-16T08:19:37.636 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-03-16T08:19:37.637 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-03-16T08:19:37.638 INFO:tasks.workunit.client.0.smithi049.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-03-16T08:19:37.639 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-03-16T08:19:37.640 INFO:tasks.workunit.client.0.smithi049.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-03-16T08:19:37.641 INFO:tasks.workunit.client.0.smithi049.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-03-16T08:19:37.642 INFO:tasks.workunit.client.0.smithi049.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:khugepage 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:khugepage 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:khugepage 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:khugepage 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:cryptd 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:cryptd 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:cryptd 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-03-16T08:19:37.643 INFO:tasks.workunit.client.0.smithi049.stdout:cryptd 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kintegrit 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kintegrit 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kintegrit 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kintegrit 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kblockd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kblockd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kblockd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:kblockd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:blkcg_pun 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:blkcg_pun 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:blkcg_pun 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-03-16T08:19:37.644 INFO:tasks.workunit.client.0.smithi049.stdout:blkcg_pun 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:tpm_dev_w 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:tpm_dev_w 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:tpm_dev_w 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:tpm_dev_w 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md_bitmap 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md_bitmap 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md_bitmap 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-03-16T08:19:37.645 INFO:tasks.workunit.client.0.smithi049.stdout:md_bitmap 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:edac-poll 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:edac-poll 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:edac-poll 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:edac-poll 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:watchdogd 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:watchdogd 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:watchdogd 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:watchdogd 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-03-16T08:19:37.646 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kswapd0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kswapd0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kswapd0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kswapd0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kthrotld 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kthrotld 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kthrotld 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:kthrotld 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:acpi_ther 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-03-16T08:19:37.647 INFO:tasks.workunit.client.0.smithi049.stdout:acpi_ther 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:acpi_ther 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:acpi_ther 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kmpath_rd 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kmpath_rd 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kmpath_rd 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kmpath_rd 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kaluad 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kaluad 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kaluad 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:kaluad 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:mld 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-03-16T08:19:37.648 INFO:tasks.workunit.client.0.smithi049.stdout:mld 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:mld 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:mld 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:ipv6_addr 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:ipv6_addr 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:ipv6_addr 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:ipv6_addr 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kstrp 125 root cwd unknown /proc/125/cwd (readlink: Permission denied) 2024-03-16T08:19:37.649 INFO:tasks.workunit.client.0.smithi049.stdout:kstrp 125 root rtd unknown /proc/125/root (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kstrp 125 root txt unknown /proc/125/exe (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kstrp 125 root NOFD /proc/125/fd (opendir: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 127 root cwd unknown /proc/127/cwd (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 127 root rtd unknown /proc/127/root (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 127 root txt unknown /proc/127/exe (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 127 root NOFD /proc/127/fd (opendir: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:zswap-shr 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:zswap-shr 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:zswap-shr 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-03-16T08:19:37.650 INFO:tasks.workunit.client.0.smithi049.stdout:zswap-shr 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 133 root cwd unknown /proc/133/cwd (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 133 root rtd unknown /proc/133/root (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 133 root txt unknown /proc/133/exe (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 133 root NOFD /proc/133/fd (opendir: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 278 root cwd unknown /proc/278/cwd (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 278 root rtd unknown /proc/278/root (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 278 root txt unknown /proc/278/exe (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 278 root NOFD /proc/278/fd (opendir: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 280 root cwd unknown /proc/280/cwd (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 280 root rtd unknown /proc/280/root (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 280 root txt unknown /proc/280/exe (readlink: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 280 root NOFD /proc/280/fd (opendir: Permission denied) 2024-03-16T08:19:37.651 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 283 root cwd unknown /proc/283/cwd (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 283 root rtd unknown /proc/283/root (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 283 root txt unknown /proc/283/exe (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 283 root NOFD /proc/283/fd (opendir: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 292 root cwd unknown /proc/292/cwd (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 292 root rtd unknown /proc/292/root (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 292 root txt unknown /proc/292/exe (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 292 root NOFD /proc/292/fd (opendir: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 294 root cwd unknown /proc/294/cwd (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 294 root rtd unknown /proc/294/root (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 294 root txt unknown /proc/294/exe (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 294 root NOFD /proc/294/fd (opendir: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 310 root cwd unknown /proc/310/cwd (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 310 root rtd unknown /proc/310/root (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 310 root txt unknown /proc/310/exe (readlink: Permission denied) 2024-03-16T08:19:37.652 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 310 root NOFD /proc/310/fd (opendir: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 386 root cwd unknown /proc/386/cwd (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 386 root rtd unknown /proc/386/root (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 386 root txt unknown /proc/386/exe (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 386 root NOFD /proc/386/fd (opendir: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-wq 419 root cwd unknown /proc/419/cwd (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-wq 419 root rtd unknown /proc/419/root (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-wq 419 root txt unknown /proc/419/exe (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-wq 419 root NOFD /proc/419/fd (opendir: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-rese 420 root cwd unknown /proc/420/cwd (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-rese 420 root rtd unknown /proc/420/root (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-rese 420 root txt unknown /proc/420/exe (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-rese 420 root NOFD /proc/420/fd (opendir: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-dele 421 root cwd unknown /proc/421/cwd (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-dele 421 root rtd unknown /proc/421/root (readlink: Permission denied) 2024-03-16T08:19:37.653 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-dele 421 root txt unknown /proc/421/exe (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-dele 421 root NOFD /proc/421/fd (opendir: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-auth 422 root cwd unknown /proc/422/cwd (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-auth 422 root rtd unknown /proc/422/root (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-auth 422 root txt unknown /proc/422/exe (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:nvme-auth 422 root NOFD /proc/422/fd (opendir: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ata_sff 425 root cwd unknown /proc/425/cwd (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ata_sff 425 root rtd unknown /proc/425/root (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ata_sff 425 root txt unknown /proc/425/exe (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ata_sff 425 root NOFD /proc/425/fd (opendir: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ixgbe 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ixgbe 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ixgbe 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:ixgbe 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-03-16T08:19:37.654 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_0 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_0 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_0 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_0 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_1 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_1 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_1 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_1 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-03-16T08:19:37.655 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_2 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_2 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_2 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_2 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_3 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_3 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_3 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_3 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-03-16T08:19:37.656 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_4 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_4 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_4 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_4 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_5 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_5 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_5 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_5 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-03-16T08:19:37.657 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_6 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_6 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_6 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_6 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_7 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-03-16T08:19:37.658 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_7 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_7 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_7 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_8 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_8 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_8 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_8 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-03-16T08:19:37.659 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_9 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_9 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_9 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_eh_9 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 471 root cwd unknown /proc/471/cwd (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 471 root rtd unknown /proc/471/root (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 471 root txt unknown /proc/471/exe (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:scsi_tmf_ 471 root NOFD /proc/471/fd (opendir: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 472 root cwd unknown /proc/472/cwd (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 472 root rtd unknown /proc/472/root (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 472 root txt unknown /proc/472/exe (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 472 root NOFD /proc/472/fd (opendir: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:jbd2/sda1 499 root cwd unknown /proc/499/cwd (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:jbd2/sda1 499 root rtd unknown /proc/499/root (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:jbd2/sda1 499 root txt unknown /proc/499/exe (readlink: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:jbd2/sda1 499 root NOFD /proc/499/fd (opendir: Permission denied) 2024-03-16T08:19:37.660 INFO:tasks.workunit.client.0.smithi049.stdout:ext4-rsv- 500 root cwd unknown /proc/500/cwd (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4-rsv- 500 root rtd unknown /proc/500/root (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4-rsv- 500 root txt unknown /proc/500/exe (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4-rsv- 500 root NOFD /proc/500/fd (opendir: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-j 575 root cwd unknown /proc/575/cwd (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-j 575 root rtd unknown /proc/575/root (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-j 575 root txt unknown /proc/575/exe (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-j 575 root NOFD /proc/575/fd (opendir: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4lazyi 581 root cwd unknown /proc/581/cwd (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4lazyi 581 root rtd unknown /proc/581/root (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4lazyi 581 root txt unknown /proc/581/exe (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:ext4lazyi 581 root NOFD /proc/581/fd (opendir: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 583 root cwd unknown /proc/583/cwd (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 583 root rtd unknown /proc/583/root (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 583 root txt unknown /proc/583/exe (readlink: Permission denied) 2024-03-16T08:19:37.661 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 583 root NOFD /proc/583/fd (opendir: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-u 591 root cwd unknown /proc/591/cwd (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-u 591 root rtd unknown /proc/591/root (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-u 591 root txt unknown /proc/591/exe (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-u 591 root NOFD /proc/591/fd (opendir: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 592 root cwd unknown /proc/592/cwd (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 592 root rtd unknown /proc/592/root (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 592 root txt unknown /proc/592/exe (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 592 root NOFD /proc/592/fd (opendir: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:ipmi-msgh 641 root cwd unknown /proc/641/cwd (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:ipmi-msgh 641 root rtd unknown /proc/641/root (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:ipmi-msgh 641 root txt unknown /proc/641/exe (readlink: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:ipmi-msgh 641 root NOFD /proc/641/fd (opendir: Permission denied) 2024-03-16T08:19:37.662 INFO:tasks.workunit.client.0.smithi049.stdout:kipmi0 649 root cwd unknown /proc/649/cwd (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:kipmi0 649 root rtd unknown /proc/649/root (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:kipmi0 649 root txt unknown /proc/649/exe (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:kipmi0 649 root NOFD /proc/649/fd (opendir: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpcbind 662 rpc cwd unknown /proc/662/cwd (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpcbind 662 rpc rtd unknown /proc/662/root (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpcbind 662 rpc txt unknown /proc/662/exe (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpcbind 662 rpc NOFD /proc/662/fd (opendir: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpciod 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpciod 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpciod 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:rpciod 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:xprtiod 670 root cwd unknown /proc/670/cwd (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:xprtiod 670 root rtd unknown /proc/670/root (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:xprtiod 670 root txt unknown /proc/670/exe (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:xprtiod 670 root NOFD /proc/670/fd (opendir: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 root cwd unknown /proc/671/cwd (readlink: Permission denied) 2024-03-16T08:19:37.663 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 root rtd unknown /proc/671/root (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 root txt unknown /proc/671/exe (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 root NOFD /proc/671/fd (opendir: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 672 auditd root cwd unknown /proc/671/task/672/cwd (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 672 auditd root rtd unknown /proc/671/task/672/root (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 672 auditd root txt unknown /proc/671/task/672/exe (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 672 auditd root NOFD /proc/671/task/672/fd (opendir: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 674 auditd root cwd unknown /proc/671/task/674/cwd (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 674 auditd root rtd unknown /proc/671/task/674/root (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 674 auditd root txt unknown /proc/671/task/674/exe (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:auditd 671 674 auditd root NOFD /proc/671/task/674/fd (opendir: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:sedispatc 673 root cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:sedispatc 673 root rtd unknown /proc/673/root (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:sedispatc 673 root txt unknown /proc/673/exe (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:sedispatc 673 root NOFD /proc/673/fd (opendir: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 712 dbus cwd unknown /proc/712/cwd (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 712 dbus rtd unknown /proc/712/root (readlink: Permission denied) 2024-03-16T08:19:37.664 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 712 dbus txt unknown /proc/712/exe (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 712 dbus NOFD /proc/712/fd (opendir: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 713 dbus cwd unknown /proc/713/cwd (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 713 dbus rtd unknown /proc/713/root (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 713 dbus txt unknown /proc/713/exe (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:dbus-brok 713 dbus NOFD /proc/713/fd (opendir: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 root cwd unknown /proc/714/cwd (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 root rtd unknown /proc/714/root (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 root txt unknown /proc/714/exe (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 root NOFD /proc/714/fd (opendir: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 742 gmain root cwd unknown /proc/714/task/742/cwd (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 742 gmain root rtd unknown /proc/714/task/742/root (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 742 gmain root txt unknown /proc/714/task/742/exe (readlink: Permission denied) 2024-03-16T08:19:37.665 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 742 gmain root NOFD /proc/714/task/742/fd (opendir: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 743 gdbus root cwd unknown /proc/714/task/743/cwd (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 743 gdbus root rtd unknown /proc/714/task/743/root (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 743 gdbus root txt unknown /proc/714/task/743/exe (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:NetworkMa 714 743 gdbus root NOFD /proc/714/task/743/fd (opendir: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 root cwd unknown /proc/717/cwd (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 root rtd unknown /proc/717/root (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 root txt unknown /proc/717/exe (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 root NOFD /proc/717/fd (opendir: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 724 gmain root cwd unknown /proc/717/task/724/cwd (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 724 gmain root rtd unknown /proc/717/task/724/root (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 724 gmain root txt unknown /proc/717/task/724/exe (readlink: Permission denied) 2024-03-16T08:19:37.666 INFO:tasks.workunit.client.0.smithi049.stdout:irqbalanc 717 724 gmain root NOFD /proc/717/task/724/fd (opendir: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:lsmd 718 libstoragemgmt cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:lsmd 718 libstoragemgmt rtd unknown /proc/718/root (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:lsmd 718 libstoragemgmt txt unknown /proc/718/exe (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:lsmd 718 libstoragemgmt NOFD /proc/718/fd (opendir: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:mcelog 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:mcelog 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:mcelog 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:mcelog 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 polkitd cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 polkitd rtd unknown /proc/720/root (readlink: Permission denied) 2024-03-16T08:19:37.667 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 polkitd txt unknown /proc/720/exe (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 polkitd NOFD /proc/720/fd (opendir: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 732 gmain polkitd cwd unknown /proc/720/task/732/cwd (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 732 gmain polkitd rtd unknown /proc/720/task/732/root (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 732 gmain polkitd txt unknown /proc/720/task/732/exe (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 732 gmain polkitd NOFD /proc/720/task/732/fd (opendir: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 733 gdbus polkitd cwd unknown /proc/720/task/733/cwd (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 733 gdbus polkitd rtd unknown /proc/720/task/733/root (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 733 gdbus polkitd txt unknown /proc/720/task/733/exe (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 733 gdbus polkitd NOFD /proc/720/task/733/fd (opendir: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 734 JS\x20Hel polkitd cwd unknown /proc/720/task/734/cwd (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 734 JS\x20Hel polkitd rtd unknown /proc/720/task/734/root (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 734 JS\x20Hel polkitd txt unknown /proc/720/task/734/exe (readlink: Permission denied) 2024-03-16T08:19:37.668 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 734 JS\x20Hel polkitd NOFD /proc/720/task/734/fd (opendir: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 735 JS\x20Hel polkitd cwd unknown /proc/720/task/735/cwd (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 735 JS\x20Hel polkitd rtd unknown /proc/720/task/735/root (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 735 JS\x20Hel polkitd txt unknown /proc/720/task/735/exe (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 735 JS\x20Hel polkitd NOFD /proc/720/task/735/fd (opendir: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 736 JS\x20Hel polkitd cwd unknown /proc/720/task/736/cwd (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 736 JS\x20Hel polkitd rtd unknown /proc/720/task/736/root (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 736 JS\x20Hel polkitd txt unknown /proc/720/task/736/exe (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 736 JS\x20Hel polkitd NOFD /proc/720/task/736/fd (opendir: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 737 JS\x20Hel polkitd cwd unknown /proc/720/task/737/cwd (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 737 JS\x20Hel polkitd rtd unknown /proc/720/task/737/root (readlink: Permission denied) 2024-03-16T08:19:37.669 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 737 JS\x20Hel polkitd txt unknown /proc/720/task/737/exe (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 737 JS\x20Hel polkitd NOFD /proc/720/task/737/fd (opendir: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 738 JS\x20Hel polkitd cwd unknown /proc/720/task/738/cwd (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 738 JS\x20Hel polkitd rtd unknown /proc/720/task/738/root (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 738 JS\x20Hel polkitd txt unknown /proc/720/task/738/exe (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 738 JS\x20Hel polkitd NOFD /proc/720/task/738/fd (opendir: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 739 JS\x20Hel polkitd cwd unknown /proc/720/task/739/cwd (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 739 JS\x20Hel polkitd rtd unknown /proc/720/task/739/root (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 739 JS\x20Hel polkitd txt unknown /proc/720/task/739/exe (readlink: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 739 JS\x20Hel polkitd NOFD /proc/720/task/739/fd (opendir: Permission denied) 2024-03-16T08:19:37.670 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 740 JS\x20Hel polkitd cwd unknown /proc/720/task/740/cwd (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 740 JS\x20Hel polkitd rtd unknown /proc/720/task/740/root (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 740 JS\x20Hel polkitd txt unknown /proc/720/task/740/exe (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 740 JS\x20Hel polkitd NOFD /proc/720/task/740/fd (opendir: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 741 JS\x20Hel polkitd cwd unknown /proc/720/task/741/cwd (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 741 JS\x20Hel polkitd rtd unknown /proc/720/task/741/root (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 741 JS\x20Hel polkitd txt unknown /proc/720/task/741/exe (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 741 JS\x20Hel polkitd NOFD /proc/720/task/741/fd (opendir: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 746 polkitd polkitd cwd unknown /proc/720/task/746/cwd (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 746 polkitd polkitd rtd unknown /proc/720/task/746/root (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 746 polkitd polkitd txt unknown /proc/720/task/746/exe (readlink: Permission denied) 2024-03-16T08:19:37.671 INFO:tasks.workunit.client.0.smithi049.stdout:polkitd 720 746 polkitd polkitd NOFD /proc/720/task/746/fd (opendir: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-l 723 root cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-l 723 root rtd unknown /proc/723/root (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-l 723 root txt unknown /proc/723/exe (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:systemd-l 723 root NOFD /proc/723/fd (opendir: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 802 root cwd unknown /proc/802/cwd (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 802 root rtd unknown /proc/802/root (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 802 root txt unknown /proc/802/exe (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 802 root NOFD /proc/802/fd (opendir: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 root cwd unknown /proc/803/cwd (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 root rtd unknown /proc/803/root (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 root txt unknown /proc/803/exe (readlink: Permission denied) 2024-03-16T08:19:37.672 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 root NOFD /proc/803/fd (opendir: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 888 tuned root cwd unknown /proc/803/task/888/cwd (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 888 tuned root rtd unknown /proc/803/task/888/root (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 888 tuned root txt unknown /proc/803/task/888/exe (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:tuned 803 888 tuned root NOFD /proc/803/task/888/fd (opendir: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 root cwd unknown /proc/807/cwd (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 root rtd unknown /proc/807/root (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 root txt unknown /proc/807/exe (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 root NOFD /proc/807/fd (opendir: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 808 gssproxy root cwd unknown /proc/807/task/808/cwd (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 808 gssproxy root rtd unknown /proc/807/task/808/root (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 808 gssproxy root txt unknown /proc/807/task/808/exe (readlink: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 808 gssproxy root NOFD /proc/807/task/808/fd (opendir: Permission denied) 2024-03-16T08:19:37.673 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 809 gssproxy root cwd unknown /proc/807/task/809/cwd (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 809 gssproxy root rtd unknown /proc/807/task/809/root (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 809 gssproxy root txt unknown /proc/807/task/809/exe (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 809 gssproxy root NOFD /proc/807/task/809/fd (opendir: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 810 gssproxy root cwd unknown /proc/807/task/810/cwd (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 810 gssproxy root rtd unknown /proc/807/task/810/root (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 810 gssproxy root txt unknown /proc/807/task/810/exe (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 810 gssproxy root NOFD /proc/807/task/810/fd (opendir: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 811 gssproxy root cwd unknown /proc/807/task/811/cwd (readlink: Permission denied) 2024-03-16T08:19:37.674 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 811 gssproxy root rtd unknown /proc/807/task/811/root (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 811 gssproxy root txt unknown /proc/807/task/811/exe (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 811 gssproxy root NOFD /proc/807/task/811/fd (opendir: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 812 gssproxy root cwd unknown /proc/807/task/812/cwd (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 812 gssproxy root rtd unknown /proc/807/task/812/root (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 812 gssproxy root txt unknown /proc/807/task/812/exe (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:gssproxy 807 812 gssproxy root NOFD /proc/807/task/812/fd (opendir: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:atd 814 root cwd unknown /proc/814/cwd (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:atd 814 root rtd unknown /proc/814/root (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:atd 814 root txt unknown /proc/814/exe (readlink: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:atd 814 root NOFD /proc/814/fd (opendir: Permission denied) 2024-03-16T08:19:37.675 INFO:tasks.workunit.client.0.smithi049.stdout:crond 815 root cwd unknown /proc/815/cwd (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:crond 815 root rtd unknown /proc/815/root (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:crond 815 root txt unknown /proc/815/exe (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:crond 815 root NOFD /proc/815/fd (opendir: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 909 root cwd unknown /proc/909/cwd (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 909 root rtd unknown /proc/909/root (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 909 root txt unknown /proc/909/exe (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 909 root NOFD /proc/909/fd (opendir: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:nrpe 910 nrpe cwd unknown /proc/910/cwd (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:nrpe 910 nrpe rtd unknown /proc/910/root (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:nrpe 910 nrpe txt unknown /proc/910/exe (readlink: Permission denied) 2024-03-16T08:19:37.676 INFO:tasks.workunit.client.0.smithi049.stdout:nrpe 910 nrpe NOFD /proc/910/fd (opendir: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1383 root cwd unknown /proc/1383/cwd (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1383 root rtd unknown /proc/1383/root (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1383 root txt unknown /proc/1383/exe (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1383 root NOFD /proc/1383/fd (opendir: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1393 root cwd unknown /proc/1393/cwd (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1393 root rtd unknown /proc/1393/root (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1393 root txt unknown /proc/1393/exe (readlink: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:agetty 1393 root NOFD /proc/1393/fd (opendir: Permission denied) 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu cwd DIR 8,1 4096 2 / 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu rtd DIR 8,1 4096 2 / 2024-03-16T08:19:37.677 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 1698 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 1027 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 10007 /usr/lib64/libcrypto.so.3.0.7 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2024-03-16T08:19:37.678 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 10126 /usr/lib64/libblkid.so.1.1.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 10156 /usr/lib64/libmount.so.1.1.0 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2024-03-16T08:19:37.679 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2024-03-16T08:19:37.680 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 1u unix 0x0000000000000000 0t0 19371 type=STREAM (CONNECTED) 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 2u unix 0x0000000000000000 0t0 19371 type=STREAM (CONNECTED) 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 3u unix 0x0000000000000000 0t0 19399 type=DGRAM (CONNECTED) 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 4u a_inode 0,14 0 1085 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 5u a_inode 0,14 0 1085 [signalfd] 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 6r a_inode 0,14 0 1085 inotify 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 7r DIR 0,25 0 3393 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 8u a_inode 0,14 0 1085 [timerfd] 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 9u a_inode 0,14 0 1085 [eventpoll:10,13] 2024-03-16T08:19:37.681 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 10r REG 0,20 0 19388 /proc/4854/mountinfo 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 11r a_inode 0,14 0 1085 inotify 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 12u unix 0x0000000000000000 0t0 19427 /run/user/1000/bus type=STREAM (LISTEN) 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 13r a_inode 0,14 0 1085 inotify 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 15u netlink 0t0 19409 KOBJECT_UEVENT 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 16u unix 0x0000000000000000 0t0 19414 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 17u unix 0x0000000000000000 0t0 19415 type=DGRAM (CONNECTED) 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 18u unix 0x0000000000000000 0t0 19416 type=DGRAM (CONNECTED) 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 19u unix 0x0000000000000000 0t0 19417 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-03-16T08:19:37.682 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 20u unix 0x0000000000000000 0t0 19419 type=STREAM (CONNECTED) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 4854 ubuntu 21u a_inode 0,14 0 1085 [timerfd] 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 4856 ubuntu cwd unknown /proc/4856/cwd (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 4856 ubuntu rtd unknown /proc/4856/root (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 4856 ubuntu txt unknown /proc/4856/exe (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 4856 ubuntu NOFD /proc/4856/fd (opendir: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 root cwd unknown /proc/5099/cwd (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 root rtd unknown /proc/5099/root (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 root txt unknown /proc/5099/exe (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 root NOFD /proc/5099/fd (opendir: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5101 in:imjour root cwd unknown /proc/5099/task/5101/cwd (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5101 in:imjour root rtd unknown /proc/5099/task/5101/root (readlink: Permission denied) 2024-03-16T08:19:37.683 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5101 in:imjour root txt unknown /proc/5099/task/5101/exe (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5101 in:imjour root NOFD /proc/5099/task/5101/fd (opendir: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5102 rs:main root cwd unknown /proc/5099/task/5102/cwd (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5102 rs:main root rtd unknown /proc/5099/task/5102/root (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5102 rs:main root txt unknown /proc/5099/task/5102/exe (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:rsyslogd 5099 5102 rs:main root NOFD /proc/5099/task/5102/fd (opendir: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:anacron 15642 root cwd unknown /proc/15642/cwd (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:anacron 15642 root rtd unknown /proc/15642/root (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:anacron 15642 root txt unknown /proc/15642/exe (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:anacron 15642 root NOFD /proc/15642/fd (opendir: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:tls-strp 15819 root cwd unknown /proc/15819/cwd (readlink: Permission denied) 2024-03-16T08:19:37.684 INFO:tasks.workunit.client.0.smithi049.stdout:tls-strp 15819 root rtd unknown /proc/15819/root (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:tls-strp 15819 root txt unknown /proc/15819/exe (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:tls-strp 15819 root NOFD /proc/15819/fd (opendir: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:smartd 15868 root cwd unknown /proc/15868/cwd (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:smartd 15868 root rtd unknown /proc/15868/root (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:smartd 15868 root txt unknown /proc/15868/exe (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:smartd 15868 root NOFD /proc/15868/fd (opendir: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 20598 root cwd unknown /proc/20598/cwd (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 20598 root rtd unknown /proc/20598/root (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 20598 root txt unknown /proc/20598/exe (readlink: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 20598 root NOFD /proc/20598/fd (opendir: Permission denied) 2024-03-16T08:19:37.685 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21228 root cwd unknown /proc/21228/cwd (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21228 root rtd unknown /proc/21228/root (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21228 root txt unknown /proc/21228/exe (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21228 root NOFD /proc/21228/fd (opendir: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21257 root cwd unknown /proc/21257/cwd (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21257 root rtd unknown /proc/21257/root (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21257 root txt unknown /proc/21257/exe (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21257 root NOFD /proc/21257/fd (opendir: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21286 root cwd unknown /proc/21286/cwd (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21286 root rtd unknown /proc/21286/root (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21286 root txt unknown /proc/21286/exe (readlink: Permission denied) 2024-03-16T08:19:37.686 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21286 root NOFD /proc/21286/fd (opendir: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21315 root cwd unknown /proc/21315/cwd (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21315 root rtd unknown /proc/21315/root (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21315 root txt unknown /proc/21315/exe (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21315 root NOFD /proc/21315/fd (opendir: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21344 root cwd unknown /proc/21344/cwd (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21344 root rtd unknown /proc/21344/root (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21344 root txt unknown /proc/21344/exe (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:kdmflush/ 21344 root NOFD /proc/21344/fd (opendir: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:xfsalloc 21508 root cwd unknown /proc/21508/cwd (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:xfsalloc 21508 root rtd unknown /proc/21508/root (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:xfsalloc 21508 root txt unknown /proc/21508/exe (readlink: Permission denied) 2024-03-16T08:19:37.687 INFO:tasks.workunit.client.0.smithi049.stdout:xfsalloc 21508 root NOFD /proc/21508/fd (opendir: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs_mru_c 21509 root cwd unknown /proc/21509/cwd (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs_mru_c 21509 root rtd unknown /proc/21509/root (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs_mru_c 21509 root txt unknown /proc/21509/exe (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs_mru_c 21509 root NOFD /proc/21509/fd (opendir: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-buf/d 21510 root cwd unknown /proc/21510/cwd (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-buf/d 21510 root rtd unknown /proc/21510/root (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-buf/d 21510 root txt unknown /proc/21510/exe (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-buf/d 21510 root NOFD /proc/21510/fd (opendir: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-conv/ 21511 root cwd unknown /proc/21511/cwd (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-conv/ 21511 root rtd unknown /proc/21511/root (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-conv/ 21511 root txt unknown /proc/21511/exe (readlink: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-conv/ 21511 root NOFD /proc/21511/fd (opendir: Permission denied) 2024-03-16T08:19:37.688 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-recla 21512 root cwd unknown /proc/21512/cwd (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-recla 21512 root rtd unknown /proc/21512/root (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-recla 21512 root txt unknown /proc/21512/exe (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-recla 21512 root NOFD /proc/21512/fd (opendir: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-block 21513 root cwd unknown /proc/21513/cwd (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-block 21513 root rtd unknown /proc/21513/root (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-block 21513 root txt unknown /proc/21513/exe (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-block 21513 root NOFD /proc/21513/fd (opendir: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-inode 21514 root cwd unknown /proc/21514/cwd (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-inode 21514 root rtd unknown /proc/21514/root (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-inode 21514 root txt unknown /proc/21514/exe (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-inode 21514 root NOFD /proc/21514/fd (opendir: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-log/d 21515 root cwd unknown /proc/21515/cwd (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-log/d 21515 root rtd unknown /proc/21515/root (readlink: Permission denied) 2024-03-16T08:19:37.689 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-log/d 21515 root txt unknown /proc/21515/exe (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-log/d 21515 root NOFD /proc/21515/fd (opendir: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-cil/d 21516 root cwd unknown /proc/21516/cwd (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-cil/d 21516 root rtd unknown /proc/21516/root (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-cil/d 21516 root txt unknown /proc/21516/exe (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfs-cil/d 21516 root NOFD /proc/21516/fd (opendir: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfsaild/d 21517 root cwd unknown /proc/21517/cwd (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfsaild/d 21517 root rtd unknown /proc/21517/root (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfsaild/d 21517 root txt unknown /proc/21517/exe (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:xfsaild/d 21517 root NOFD /proc/21517/fd (opendir: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23435 root cwd unknown /proc/23435/cwd (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23435 root rtd unknown /proc/23435/root (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23435 root txt unknown /proc/23435/exe (readlink: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23435 root NOFD /proc/23435/fd (opendir: Permission denied) 2024-03-16T08:19:37.690 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23439 ubuntu cwd unknown /proc/23439/cwd (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23439 ubuntu rtd unknown /proc/23439/root (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23439 ubuntu txt unknown /proc/23439/exe (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 23439 ubuntu NOFD /proc/23439/fd (opendir: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:chronyd 23499 chrony cwd unknown /proc/23499/cwd (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:chronyd 23499 chrony rtd unknown /proc/23499/root (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:chronyd 23499 chrony txt unknown /proc/23499/exe (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:chronyd 23499 chrony NOFD /proc/23499/fd (opendir: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-cras 23802 ceph cwd unknown /proc/23802/cwd (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-cras 23802 ceph rtd unknown /proc/23802/root (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-cras 23802 ceph txt unknown /proc/23802/exe (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-cras 23802 ceph NOFD /proc/23802/fd (opendir: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 24328 root cwd unknown /proc/24328/cwd (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 24328 root rtd unknown /proc/24328/root (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 24328 root txt unknown /proc/24328/exe (readlink: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 24328 root NOFD /proc/24328/fd (opendir: Permission denied) 2024-03-16T08:19:37.691 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25268 root cwd unknown /proc/25268/cwd (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25268 root rtd unknown /proc/25268/root (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25268 root txt unknown /proc/25268/exe (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25268 root NOFD /proc/25268/fd (opendir: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25269 root cwd unknown /proc/25269/cwd (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25269 root rtd unknown /proc/25269/root (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25269 root txt unknown /proc/25269/exe (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 25269 root NOFD /proc/25269/fd (opendir: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30911 root cwd unknown /proc/30911/cwd (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30911 root rtd unknown /proc/30911/root (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30911 root txt unknown /proc/30911/exe (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30911 root NOFD /proc/30911/fd (opendir: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30928 root cwd unknown /proc/30928/cwd (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30928 root rtd unknown /proc/30928/root (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30928 root txt unknown /proc/30928/exe (readlink: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30928 root NOFD /proc/30928/fd (opendir: Permission denied) 2024-03-16T08:19:37.692 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30929 root cwd unknown /proc/30929/cwd (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30929 root rtd unknown /proc/30929/root (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30929 root txt unknown /proc/30929/exe (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 30929 root NOFD /proc/30929/fd (opendir: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30946 root cwd unknown /proc/30946/cwd (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30946 root rtd unknown /proc/30946/root (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30946 root txt unknown /proc/30946/exe (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 30946 root NOFD /proc/30946/fd (opendir: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 31938 root cwd unknown /proc/31938/cwd (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 31938 root rtd unknown /proc/31938/root (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 31938 root txt unknown /proc/31938/exe (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 31938 root NOFD /proc/31938/fd (opendir: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 31940 root cwd unknown /proc/31940/cwd (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 31940 root rtd unknown /proc/31940/root (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 31940 root txt unknown /proc/31940/exe (readlink: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 31940 root NOFD /proc/31940/fd (opendir: Permission denied) 2024-03-16T08:19:37.693 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 ceph cwd unknown /proc/31942/cwd (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 ceph rtd unknown /proc/31942/root (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 ceph txt unknown /proc/31942/exe (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 ceph NOFD /proc/31942/fd (opendir: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31943 log ceph cwd unknown /proc/31942/task/31943/cwd (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31943 log ceph rtd unknown /proc/31942/task/31943/root (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31943 log ceph txt unknown /proc/31942/task/31943/exe (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31943 log ceph NOFD /proc/31942/task/31943/fd (opendir: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31944 service ceph cwd unknown /proc/31942/task/31944/cwd (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31944 service ceph rtd unknown /proc/31942/task/31944/root (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31944 service ceph txt unknown /proc/31942/task/31944/exe (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31944 service ceph NOFD /proc/31942/task/31944/fd (opendir: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31945 admin_soc ceph cwd unknown /proc/31942/task/31945/cwd (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31945 admin_soc ceph rtd unknown /proc/31942/task/31945/root (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31945 admin_soc ceph txt unknown /proc/31942/task/31945/exe (readlink: Permission denied) 2024-03-16T08:19:37.694 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31945 admin_soc ceph NOFD /proc/31942/task/31945/fd (opendir: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31946 signal_ha ceph cwd unknown /proc/31942/task/31946/cwd (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31946 signal_ha ceph rtd unknown /proc/31942/task/31946/root (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31946 signal_ha ceph txt unknown /proc/31942/task/31946/exe (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31946 signal_ha ceph NOFD /proc/31942/task/31946/fd (opendir: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31947 rocksdb:l ceph cwd unknown /proc/31942/task/31947/cwd (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31947 rocksdb:l ceph rtd unknown /proc/31942/task/31947/root (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31947 rocksdb:l ceph txt unknown /proc/31942/task/31947/exe (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31947 rocksdb:l ceph NOFD /proc/31942/task/31947/fd (opendir: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31948 rocksdb:h ceph cwd unknown /proc/31942/task/31948/cwd (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31948 rocksdb:h ceph rtd unknown /proc/31942/task/31948/root (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31948 rocksdb:h ceph txt unknown /proc/31942/task/31948/exe (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31948 rocksdb:h ceph NOFD /proc/31942/task/31948/fd (opendir: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31964 ceph-mon ceph cwd unknown /proc/31942/task/31964/cwd (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31964 ceph-mon ceph rtd unknown /proc/31942/task/31964/root (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31964 ceph-mon ceph txt unknown /proc/31942/task/31964/exe (readlink: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31964 ceph-mon ceph NOFD /proc/31942/task/31964/fd (opendir: Permission denied) 2024-03-16T08:19:37.695 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31965 fn_monsto ceph cwd unknown /proc/31942/task/31965/cwd (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31965 fn_monsto ceph rtd unknown /proc/31942/task/31965/root (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31965 fn_monsto ceph txt unknown /proc/31942/task/31965/exe (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31965 fn_monsto ceph NOFD /proc/31942/task/31965/fd (opendir: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31966 msgr-work ceph cwd unknown /proc/31942/task/31966/cwd (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31966 msgr-work ceph rtd unknown /proc/31942/task/31966/root (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31966 msgr-work ceph txt unknown /proc/31942/task/31966/exe (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31966 msgr-work ceph NOFD /proc/31942/task/31966/fd (opendir: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31967 msgr-work ceph cwd unknown /proc/31942/task/31967/cwd (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31967 msgr-work ceph rtd unknown /proc/31942/task/31967/root (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31967 msgr-work ceph txt unknown /proc/31942/task/31967/exe (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31967 msgr-work ceph NOFD /proc/31942/task/31967/fd (opendir: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31968 msgr-work ceph cwd unknown /proc/31942/task/31968/cwd (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31968 msgr-work ceph rtd unknown /proc/31942/task/31968/root (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31968 msgr-work ceph txt unknown /proc/31942/task/31968/exe (readlink: Permission denied) 2024-03-16T08:19:37.696 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31968 msgr-work ceph NOFD /proc/31942/task/31968/fd (opendir: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31969 OpHistory ceph cwd unknown /proc/31942/task/31969/cwd (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31969 OpHistory ceph rtd unknown /proc/31942/task/31969/root (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31969 OpHistory ceph txt unknown /proc/31942/task/31969/exe (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31969 OpHistory ceph NOFD /proc/31942/task/31969/fd (opendir: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31970 fin ceph cwd unknown /proc/31942/task/31970/cwd (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31970 fin ceph rtd unknown /proc/31942/task/31970/root (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31970 fin ceph txt unknown /proc/31942/task/31970/exe (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31970 fin ceph NOFD /proc/31942/task/31970/fd (opendir: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31971 safe_time ceph cwd unknown /proc/31942/task/31971/cwd (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31971 safe_time ceph rtd unknown /proc/31942/task/31971/root (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31971 safe_time ceph txt unknown /proc/31942/task/31971/exe (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31971 safe_time ceph NOFD /proc/31942/task/31971/fd (opendir: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31972 cpu_tp ceph cwd unknown /proc/31942/task/31972/cwd (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31972 cpu_tp ceph rtd unknown /proc/31942/task/31972/root (readlink: Permission denied) 2024-03-16T08:19:37.697 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31972 cpu_tp ceph txt unknown /proc/31942/task/31972/exe (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31972 cpu_tp ceph NOFD /proc/31942/task/31972/fd (opendir: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31973 cpu_tp ceph cwd unknown /proc/31942/task/31973/cwd (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31973 cpu_tp ceph rtd unknown /proc/31942/task/31973/root (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31973 cpu_tp ceph txt unknown /proc/31942/task/31973/exe (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31973 cpu_tp ceph NOFD /proc/31942/task/31973/fd (opendir: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31974 cpu_tp ceph cwd unknown /proc/31942/task/31974/cwd (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31974 cpu_tp ceph rtd unknown /proc/31942/task/31974/root (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31974 cpu_tp ceph txt unknown /proc/31942/task/31974/exe (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31974 cpu_tp ceph NOFD /proc/31942/task/31974/fd (opendir: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31975 cpu_tp ceph cwd unknown /proc/31942/task/31975/cwd (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31975 cpu_tp ceph rtd unknown /proc/31942/task/31975/root (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31975 cpu_tp ceph txt unknown /proc/31942/task/31975/exe (readlink: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31975 cpu_tp ceph NOFD /proc/31942/task/31975/fd (opendir: Permission denied) 2024-03-16T08:19:37.698 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31976 ms_dispat ceph cwd unknown /proc/31942/task/31976/cwd (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31976 ms_dispat ceph rtd unknown /proc/31942/task/31976/root (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31976 ms_dispat ceph txt unknown /proc/31942/task/31976/exe (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31976 ms_dispat ceph NOFD /proc/31942/task/31976/fd (opendir: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31977 ms_local ceph cwd unknown /proc/31942/task/31977/cwd (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31977 ms_local ceph rtd unknown /proc/31942/task/31977/root (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31977 ms_local ceph txt unknown /proc/31942/task/31977/exe (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31977 ms_local ceph NOFD /proc/31942/task/31977/fd (opendir: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31978 safe_time ceph cwd unknown /proc/31942/task/31978/cwd (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31978 safe_time ceph rtd unknown /proc/31942/task/31978/root (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31978 safe_time ceph txt unknown /proc/31942/task/31978/exe (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31978 safe_time ceph NOFD /proc/31942/task/31978/fd (opendir: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31979 ms_dispat ceph cwd unknown /proc/31942/task/31979/cwd (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31979 ms_dispat ceph rtd unknown /proc/31942/task/31979/root (readlink: Permission denied) 2024-03-16T08:19:37.699 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31979 ms_dispat ceph txt unknown /proc/31942/task/31979/exe (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31979 ms_dispat ceph NOFD /proc/31942/task/31979/fd (opendir: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31980 ms_local ceph cwd unknown /proc/31942/task/31980/cwd (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31980 ms_local ceph rtd unknown /proc/31942/task/31980/root (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31980 ms_local ceph txt unknown /proc/31942/task/31980/exe (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 31980 ms_local ceph NOFD /proc/31942/task/31980/fd (opendir: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 70772 rstore_co ceph cwd unknown /proc/31942/task/70772/cwd (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 70772 rstore_co ceph rtd unknown /proc/31942/task/70772/root (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 70772 rstore_co ceph txt unknown /proc/31942/task/70772/exe (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mon 31942 70772 rstore_co ceph NOFD /proc/31942/task/70772/fd (opendir: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 32175 root cwd unknown /proc/32175/cwd (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 32175 root rtd unknown /proc/32175/root (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 32175 root txt unknown /proc/32175/exe (readlink: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 32175 root NOFD /proc/32175/fd (opendir: Permission denied) 2024-03-16T08:19:37.700 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 32177 root cwd unknown /proc/32177/cwd (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 32177 root rtd unknown /proc/32177/root (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 32177 root txt unknown /proc/32177/exe (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 32177 root NOFD /proc/32177/fd (opendir: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 ceph cwd unknown /proc/32179/cwd (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 ceph rtd unknown /proc/32179/root (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 ceph txt unknown /proc/32179/exe (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 ceph NOFD /proc/32179/fd (opendir: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84477 log ceph cwd unknown /proc/32179/task/84477/cwd (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84477 log ceph rtd unknown /proc/32179/task/84477/root (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84477 log ceph txt unknown /proc/32179/task/84477/exe (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84477 log ceph NOFD /proc/32179/task/84477/fd (opendir: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84479 msgr-work ceph cwd unknown /proc/32179/task/84479/cwd (readlink: Permission denied) 2024-03-16T08:19:37.701 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84479 msgr-work ceph rtd unknown /proc/32179/task/84479/root (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84479 msgr-work ceph txt unknown /proc/32179/task/84479/exe (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84479 msgr-work ceph NOFD /proc/32179/task/84479/fd (opendir: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84480 msgr-work ceph cwd unknown /proc/32179/task/84480/cwd (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84480 msgr-work ceph rtd unknown /proc/32179/task/84480/root (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84480 msgr-work ceph txt unknown /proc/32179/task/84480/exe (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84480 msgr-work ceph NOFD /proc/32179/task/84480/fd (opendir: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84481 msgr-work ceph cwd unknown /proc/32179/task/84481/cwd (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84481 msgr-work ceph rtd unknown /proc/32179/task/84481/root (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84481 msgr-work ceph txt unknown /proc/32179/task/84481/exe (readlink: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84481 msgr-work ceph NOFD /proc/32179/task/84481/fd (opendir: Permission denied) 2024-03-16T08:19:37.702 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84485 service ceph cwd unknown /proc/32179/task/84485/cwd (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84485 service ceph rtd unknown /proc/32179/task/84485/root (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84485 service ceph txt unknown /proc/32179/task/84485/exe (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84485 service ceph NOFD /proc/32179/task/84485/fd (opendir: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84486 admin_soc ceph cwd unknown /proc/32179/task/84486/cwd (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84486 admin_soc ceph rtd unknown /proc/32179/task/84486/root (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84486 admin_soc ceph txt unknown /proc/32179/task/84486/exe (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84486 admin_soc ceph NOFD /proc/32179/task/84486/fd (opendir: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84487 ceph_time ceph cwd unknown /proc/32179/task/84487/cwd (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84487 ceph_time ceph rtd unknown /proc/32179/task/84487/root (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84487 ceph_time ceph txt unknown /proc/32179/task/84487/exe (readlink: Permission denied) 2024-03-16T08:19:37.703 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84487 ceph_time ceph NOFD /proc/32179/task/84487/fd (opendir: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84488 fn_anonym ceph cwd unknown /proc/32179/task/84488/cwd (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84488 fn_anonym ceph rtd unknown /proc/32179/task/84488/root (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84488 fn_anonym ceph txt unknown /proc/32179/task/84488/exe (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84488 fn_anonym ceph NOFD /proc/32179/task/84488/fd (opendir: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84489 signal_ha ceph cwd unknown /proc/32179/task/84489/cwd (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84489 signal_ha ceph rtd unknown /proc/32179/task/84489/root (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84489 signal_ha ceph txt unknown /proc/32179/task/84489/exe (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84489 signal_ha ceph NOFD /proc/32179/task/84489/fd (opendir: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84490 mgrsb-fin ceph cwd unknown /proc/32179/task/84490/cwd (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84490 mgrsb-fin ceph rtd unknown /proc/32179/task/84490/root (readlink: Permission denied) 2024-03-16T08:19:37.704 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84490 mgrsb-fin ceph txt unknown /proc/32179/task/84490/exe (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84490 mgrsb-fin ceph NOFD /proc/32179/task/84490/fd (opendir: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84491 ms_dispat ceph cwd unknown /proc/32179/task/84491/cwd (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84491 ms_dispat ceph rtd unknown /proc/32179/task/84491/root (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84491 ms_dispat ceph txt unknown /proc/32179/task/84491/exe (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84491 ms_dispat ceph NOFD /proc/32179/task/84491/fd (opendir: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84492 ms_local ceph cwd unknown /proc/32179/task/84492/cwd (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84492 ms_local ceph rtd unknown /proc/32179/task/84492/root (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84492 ms_local ceph txt unknown /proc/32179/task/84492/exe (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84492 ms_local ceph NOFD /proc/32179/task/84492/fd (opendir: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84493 io_contex ceph cwd unknown /proc/32179/task/84493/cwd (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84493 io_contex ceph rtd unknown /proc/32179/task/84493/root (readlink: Permission denied) 2024-03-16T08:19:37.705 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84493 io_contex ceph txt unknown /proc/32179/task/84493/exe (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84493 io_contex ceph NOFD /proc/32179/task/84493/fd (opendir: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84494 io_contex ceph cwd unknown /proc/32179/task/84494/cwd (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84494 io_contex ceph rtd unknown /proc/32179/task/84494/root (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84494 io_contex ceph txt unknown /proc/32179/task/84494/exe (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84494 io_contex ceph NOFD /proc/32179/task/84494/fd (opendir: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84495 safe_time ceph cwd unknown /proc/32179/task/84495/cwd (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84495 safe_time ceph rtd unknown /proc/32179/task/84495/root (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84495 safe_time ceph txt unknown /proc/32179/task/84495/exe (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84495 safe_time ceph NOFD /proc/32179/task/84495/fd (opendir: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84496 safe_time ceph cwd unknown /proc/32179/task/84496/cwd (readlink: Permission denied) 2024-03-16T08:19:37.706 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84496 safe_time ceph rtd unknown /proc/32179/task/84496/root (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84496 safe_time ceph txt unknown /proc/32179/task/84496/exe (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84496 safe_time ceph NOFD /proc/32179/task/84496/fd (opendir: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84497 safe_time ceph cwd unknown /proc/32179/task/84497/cwd (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84497 safe_time ceph rtd unknown /proc/32179/task/84497/root (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84497 safe_time ceph txt unknown /proc/32179/task/84497/exe (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84497 safe_time ceph NOFD /proc/32179/task/84497/fd (opendir: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84498 fn_anonym ceph cwd unknown /proc/32179/task/84498/cwd (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84498 fn_anonym ceph rtd unknown /proc/32179/task/84498/root (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84498 fn_anonym ceph txt unknown /proc/32179/task/84498/exe (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84498 fn_anonym ceph NOFD /proc/32179/task/84498/fd (opendir: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84499 flusher ceph cwd unknown /proc/32179/task/84499/cwd (readlink: Permission denied) 2024-03-16T08:19:37.707 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84499 flusher ceph rtd unknown /proc/32179/task/84499/root (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84499 flusher ceph txt unknown /proc/32179/task/84499/exe (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84499 flusher ceph NOFD /proc/32179/task/84499/fd (opendir: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84500 safe_time ceph cwd unknown /proc/32179/task/84500/cwd (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84500 safe_time ceph rtd unknown /proc/32179/task/84500/root (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84500 safe_time ceph txt unknown /proc/32179/task/84500/exe (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84500 safe_time ceph NOFD /proc/32179/task/84500/fd (opendir: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84663 mgr-fin ceph cwd unknown /proc/32179/task/84663/cwd (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84663 mgr-fin ceph rtd unknown /proc/32179/task/84663/root (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84663 mgr-fin ceph txt unknown /proc/32179/task/84663/exe (readlink: Permission denied) 2024-03-16T08:19:37.708 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84663 mgr-fin ceph NOFD /proc/32179/task/84663/fd (opendir: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84664 ms_dispat ceph cwd unknown /proc/32179/task/84664/cwd (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84664 ms_dispat ceph rtd unknown /proc/32179/task/84664/root (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84664 ms_dispat ceph txt unknown /proc/32179/task/84664/exe (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84664 ms_dispat ceph NOFD /proc/32179/task/84664/fd (opendir: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84665 ms_local ceph cwd unknown /proc/32179/task/84665/cwd (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84665 ms_local ceph rtd unknown /proc/32179/task/84665/root (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84665 ms_local ceph txt unknown /proc/32179/task/84665/exe (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84665 ms_local ceph NOFD /proc/32179/task/84665/fd (opendir: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84666 safe_time ceph cwd unknown /proc/32179/task/84666/cwd (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84666 safe_time ceph rtd unknown /proc/32179/task/84666/root (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84666 safe_time ceph txt unknown /proc/32179/task/84666/exe (readlink: Permission denied) 2024-03-16T08:19:37.709 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84666 safe_time ceph NOFD /proc/32179/task/84666/fd (opendir: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84667 cmdfin ceph cwd unknown /proc/32179/task/84667/cwd (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84667 cmdfin ceph rtd unknown /proc/32179/task/84667/root (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84667 cmdfin ceph txt unknown /proc/32179/task/84667/exe (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84667 cmdfin ceph NOFD /proc/32179/task/84667/fd (opendir: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84671 io_contex ceph cwd unknown /proc/32179/task/84671/cwd (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84671 io_contex ceph rtd unknown /proc/32179/task/84671/root (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84671 io_contex ceph txt unknown /proc/32179/task/84671/exe (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84671 io_contex ceph NOFD /proc/32179/task/84671/fd (opendir: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84672 io_contex ceph cwd unknown /proc/32179/task/84672/cwd (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84672 io_contex ceph rtd unknown /proc/32179/task/84672/root (readlink: Permission denied) 2024-03-16T08:19:37.710 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84672 io_contex ceph txt unknown /proc/32179/task/84672/exe (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84672 io_contex ceph NOFD /proc/32179/task/84672/fd (opendir: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84673 ceph_time ceph cwd unknown /proc/32179/task/84673/cwd (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84673 ceph_time ceph rtd unknown /proc/32179/task/84673/root (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84673 ceph_time ceph txt unknown /proc/32179/task/84673/exe (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84673 ceph_time ceph NOFD /proc/32179/task/84673/fd (opendir: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84674 ms_dispat ceph cwd unknown /proc/32179/task/84674/cwd (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84674 ms_dispat ceph rtd unknown /proc/32179/task/84674/root (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84674 ms_dispat ceph txt unknown /proc/32179/task/84674/exe (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84674 ms_dispat ceph NOFD /proc/32179/task/84674/fd (opendir: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84675 ms_local ceph cwd unknown /proc/32179/task/84675/cwd (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84675 ms_local ceph rtd unknown /proc/32179/task/84675/root (readlink: Permission denied) 2024-03-16T08:19:37.711 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84675 ms_local ceph txt unknown /proc/32179/task/84675/exe (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84675 ms_local ceph NOFD /proc/32179/task/84675/fd (opendir: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84676 safe_time ceph cwd unknown /proc/32179/task/84676/cwd (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84676 safe_time ceph rtd unknown /proc/32179/task/84676/root (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84676 safe_time ceph txt unknown /proc/32179/task/84676/exe (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84676 safe_time ceph NOFD /proc/32179/task/84676/fd (opendir: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84677 safe_time ceph cwd unknown /proc/32179/task/84677/cwd (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84677 safe_time ceph rtd unknown /proc/32179/task/84677/root (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84677 safe_time ceph txt unknown /proc/32179/task/84677/exe (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84677 safe_time ceph NOFD /proc/32179/task/84677/fd (opendir: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84678 balancer ceph cwd unknown /proc/32179/task/84678/cwd (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84678 balancer ceph rtd unknown /proc/32179/task/84678/root (readlink: Permission denied) 2024-03-16T08:19:37.712 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84678 balancer ceph txt unknown /proc/32179/task/84678/exe (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84678 balancer ceph NOFD /proc/32179/task/84678/fd (opendir: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84679 m-fin-bal ceph cwd unknown /proc/32179/task/84679/cwd (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84679 m-fin-bal ceph rtd unknown /proc/32179/task/84679/root (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84679 m-fin-bal ceph txt unknown /proc/32179/task/84679/exe (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84679 m-fin-bal ceph NOFD /proc/32179/task/84679/fd (opendir: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84680 mgr-fin ceph cwd unknown /proc/32179/task/84680/cwd (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84680 mgr-fin ceph rtd unknown /proc/32179/task/84680/root (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84680 mgr-fin ceph txt unknown /proc/32179/task/84680/exe (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84680 mgr-fin ceph NOFD /proc/32179/task/84680/fd (opendir: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84681 mgr-fin ceph cwd unknown /proc/32179/task/84681/cwd (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84681 mgr-fin ceph rtd unknown /proc/32179/task/84681/root (readlink: Permission denied) 2024-03-16T08:19:37.713 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84681 mgr-fin ceph txt unknown /proc/32179/task/84681/exe (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84681 mgr-fin ceph NOFD /proc/32179/task/84681/fd (opendir: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84682 mgr-fin ceph cwd unknown /proc/32179/task/84682/cwd (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84682 mgr-fin ceph rtd unknown /proc/32179/task/84682/root (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84682 mgr-fin ceph txt unknown /proc/32179/task/84682/exe (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84682 mgr-fin ceph NOFD /proc/32179/task/84682/fd (opendir: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84683 mgr-fin ceph cwd unknown /proc/32179/task/84683/cwd (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84683 mgr-fin ceph rtd unknown /proc/32179/task/84683/root (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84683 mgr-fin ceph txt unknown /proc/32179/task/84683/exe (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84683 mgr-fin ceph NOFD /proc/32179/task/84683/fd (opendir: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84684 mgr-fin ceph cwd unknown /proc/32179/task/84684/cwd (readlink: Permission denied) 2024-03-16T08:19:37.714 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84684 mgr-fin ceph rtd unknown /proc/32179/task/84684/root (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84684 mgr-fin ceph txt unknown /proc/32179/task/84684/exe (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84684 mgr-fin ceph NOFD /proc/32179/task/84684/fd (opendir: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84685 mgr-fin ceph cwd unknown /proc/32179/task/84685/cwd (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84685 mgr-fin ceph rtd unknown /proc/32179/task/84685/root (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84685 mgr-fin ceph txt unknown /proc/32179/task/84685/exe (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84685 mgr-fin ceph NOFD /proc/32179/task/84685/fd (opendir: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84686 mgr-fin ceph cwd unknown /proc/32179/task/84686/cwd (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84686 mgr-fin ceph rtd unknown /proc/32179/task/84686/root (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84686 mgr-fin ceph txt unknown /proc/32179/task/84686/exe (readlink: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84686 mgr-fin ceph NOFD /proc/32179/task/84686/fd (opendir: Permission denied) 2024-03-16T08:19:37.715 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84687 mgr-fin ceph cwd unknown /proc/32179/task/84687/cwd (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84687 mgr-fin ceph rtd unknown /proc/32179/task/84687/root (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84687 mgr-fin ceph txt unknown /proc/32179/task/84687/exe (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84687 mgr-fin ceph NOFD /proc/32179/task/84687/fd (opendir: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84688 mgr-fin ceph cwd unknown /proc/32179/task/84688/cwd (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84688 mgr-fin ceph rtd unknown /proc/32179/task/84688/root (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84688 mgr-fin ceph txt unknown /proc/32179/task/84688/exe (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84688 mgr-fin ceph NOFD /proc/32179/task/84688/fd (opendir: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84689 mgr-fin ceph cwd unknown /proc/32179/task/84689/cwd (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84689 mgr-fin ceph rtd unknown /proc/32179/task/84689/root (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84689 mgr-fin ceph txt unknown /proc/32179/task/84689/exe (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84689 mgr-fin ceph NOFD /proc/32179/task/84689/fd (opendir: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84690 mgr-fin ceph cwd unknown /proc/32179/task/84690/cwd (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84690 mgr-fin ceph rtd unknown /proc/32179/task/84690/root (readlink: Permission denied) 2024-03-16T08:19:37.716 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84690 mgr-fin ceph txt unknown /proc/32179/task/84690/exe (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84690 mgr-fin ceph NOFD /proc/32179/task/84690/fd (opendir: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84691 mgr-fin ceph cwd unknown /proc/32179/task/84691/cwd (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84691 mgr-fin ceph rtd unknown /proc/32179/task/84691/root (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84691 mgr-fin ceph txt unknown /proc/32179/task/84691/exe (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84691 mgr-fin ceph NOFD /proc/32179/task/84691/fd (opendir: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84692 mgr-fin ceph cwd unknown /proc/32179/task/84692/cwd (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84692 mgr-fin ceph rtd unknown /proc/32179/task/84692/root (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84692 mgr-fin ceph txt unknown /proc/32179/task/84692/exe (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84692 mgr-fin ceph NOFD /proc/32179/task/84692/fd (opendir: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84693 mgr-fin ceph cwd unknown /proc/32179/task/84693/cwd (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84693 mgr-fin ceph rtd unknown /proc/32179/task/84693/root (readlink: Permission denied) 2024-03-16T08:19:37.717 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84693 mgr-fin ceph txt unknown /proc/32179/task/84693/exe (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84693 mgr-fin ceph NOFD /proc/32179/task/84693/fd (opendir: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84694 mgr-fin ceph cwd unknown /proc/32179/task/84694/cwd (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84694 mgr-fin ceph rtd unknown /proc/32179/task/84694/root (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84694 mgr-fin ceph txt unknown /proc/32179/task/84694/exe (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84694 mgr-fin ceph NOFD /proc/32179/task/84694/fd (opendir: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84695 mgr-fin ceph cwd unknown /proc/32179/task/84695/cwd (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84695 mgr-fin ceph rtd unknown /proc/32179/task/84695/root (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84695 mgr-fin ceph txt unknown /proc/32179/task/84695/exe (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84695 mgr-fin ceph NOFD /proc/32179/task/84695/fd (opendir: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84696 cephadm ceph cwd unknown /proc/32179/task/84696/cwd (readlink: Permission denied) 2024-03-16T08:19:37.718 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84696 cephadm ceph rtd unknown /proc/32179/task/84696/root (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84696 cephadm ceph txt unknown /proc/32179/task/84696/exe (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84696 cephadm ceph NOFD /proc/32179/task/84696/fd (opendir: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84697 m-fin-cep ceph cwd unknown /proc/32179/task/84697/cwd (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84697 m-fin-cep ceph rtd unknown /proc/32179/task/84697/root (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84697 m-fin-cep ceph txt unknown /proc/32179/task/84697/exe (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84697 m-fin-cep ceph NOFD /proc/32179/task/84697/fd (opendir: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84698 mgr-fin ceph cwd unknown /proc/32179/task/84698/cwd (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84698 mgr-fin ceph rtd unknown /proc/32179/task/84698/root (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84698 mgr-fin ceph txt unknown /proc/32179/task/84698/exe (readlink: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84698 mgr-fin ceph NOFD /proc/32179/task/84698/fd (opendir: Permission denied) 2024-03-16T08:19:37.719 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84699 cephadm ceph cwd unknown /proc/32179/task/84699/cwd (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84699 cephadm ceph rtd unknown /proc/32179/task/84699/root (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84699 cephadm ceph txt unknown /proc/32179/task/84699/exe (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84699 cephadm ceph NOFD /proc/32179/task/84699/fd (opendir: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84700 mgr-fin ceph cwd unknown /proc/32179/task/84700/cwd (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84700 mgr-fin ceph rtd unknown /proc/32179/task/84700/root (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84700 mgr-fin ceph txt unknown /proc/32179/task/84700/exe (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84700 mgr-fin ceph NOFD /proc/32179/task/84700/fd (opendir: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84701 mgr-fin ceph cwd unknown /proc/32179/task/84701/cwd (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84701 mgr-fin ceph rtd unknown /proc/32179/task/84701/root (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84701 mgr-fin ceph txt unknown /proc/32179/task/84701/exe (readlink: Permission denied) 2024-03-16T08:19:37.720 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84701 mgr-fin ceph NOFD /proc/32179/task/84701/fd (opendir: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84702 crash ceph cwd unknown /proc/32179/task/84702/cwd (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84702 crash ceph rtd unknown /proc/32179/task/84702/root (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84702 crash ceph txt unknown /proc/32179/task/84702/exe (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84702 crash ceph NOFD /proc/32179/task/84702/fd (opendir: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84703 m-fin-cra ceph cwd unknown /proc/32179/task/84703/cwd (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84703 m-fin-cra ceph rtd unknown /proc/32179/task/84703/root (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84703 m-fin-cra ceph txt unknown /proc/32179/task/84703/exe (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84703 m-fin-cra ceph NOFD /proc/32179/task/84703/fd (opendir: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84704 mgr-fin ceph cwd unknown /proc/32179/task/84704/cwd (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84704 mgr-fin ceph rtd unknown /proc/32179/task/84704/root (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84704 mgr-fin ceph txt unknown /proc/32179/task/84704/exe (readlink: Permission denied) 2024-03-16T08:19:37.721 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84704 mgr-fin ceph NOFD /proc/32179/task/84704/fd (opendir: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84705 mgr-fin ceph cwd unknown /proc/32179/task/84705/cwd (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84705 mgr-fin ceph rtd unknown /proc/32179/task/84705/root (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84705 mgr-fin ceph txt unknown /proc/32179/task/84705/exe (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84705 mgr-fin ceph NOFD /proc/32179/task/84705/fd (opendir: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84706 mgr-fin ceph cwd unknown /proc/32179/task/84706/cwd (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84706 mgr-fin ceph rtd unknown /proc/32179/task/84706/root (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84706 mgr-fin ceph txt unknown /proc/32179/task/84706/exe (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84706 mgr-fin ceph NOFD /proc/32179/task/84706/fd (opendir: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84707 mgr-fin ceph cwd unknown /proc/32179/task/84707/cwd (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84707 mgr-fin ceph rtd unknown /proc/32179/task/84707/root (readlink: Permission denied) 2024-03-16T08:19:37.722 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84707 mgr-fin ceph txt unknown /proc/32179/task/84707/exe (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84707 mgr-fin ceph NOFD /proc/32179/task/84707/fd (opendir: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84708 dashboard ceph cwd unknown /proc/32179/task/84708/cwd (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84708 dashboard ceph rtd unknown /proc/32179/task/84708/root (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84708 dashboard ceph txt unknown /proc/32179/task/84708/exe (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84708 dashboard ceph NOFD /proc/32179/task/84708/fd (opendir: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84709 m-fin-das ceph cwd unknown /proc/32179/task/84709/cwd (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84709 m-fin-das ceph rtd unknown /proc/32179/task/84709/root (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84709 m-fin-das ceph txt unknown /proc/32179/task/84709/exe (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84709 m-fin-das ceph NOFD /proc/32179/task/84709/fd (opendir: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84710 mgr-fin ceph cwd unknown /proc/32179/task/84710/cwd (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84710 mgr-fin ceph rtd unknown /proc/32179/task/84710/root (readlink: Permission denied) 2024-03-16T08:19:37.723 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84710 mgr-fin ceph txt unknown /proc/32179/task/84710/exe (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84710 mgr-fin ceph NOFD /proc/32179/task/84710/fd (opendir: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84711 mgr-fin ceph cwd unknown /proc/32179/task/84711/cwd (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84711 mgr-fin ceph rtd unknown /proc/32179/task/84711/root (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84711 mgr-fin ceph txt unknown /proc/32179/task/84711/exe (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84711 mgr-fin ceph NOFD /proc/32179/task/84711/fd (opendir: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84712 mgr-fin ceph cwd unknown /proc/32179/task/84712/cwd (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84712 mgr-fin ceph rtd unknown /proc/32179/task/84712/root (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84712 mgr-fin ceph txt unknown /proc/32179/task/84712/exe (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84712 mgr-fin ceph NOFD /proc/32179/task/84712/fd (opendir: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84713 mgr-fin ceph cwd unknown /proc/32179/task/84713/cwd (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84713 mgr-fin ceph rtd unknown /proc/32179/task/84713/root (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84713 mgr-fin ceph txt unknown /proc/32179/task/84713/exe (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84713 mgr-fin ceph NOFD /proc/32179/task/84713/fd (opendir: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84714 devicehea ceph cwd unknown /proc/32179/task/84714/cwd (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84714 devicehea ceph rtd unknown /proc/32179/task/84714/root (readlink: Permission denied) 2024-03-16T08:19:37.724 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84714 devicehea ceph txt unknown /proc/32179/task/84714/exe (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84714 devicehea ceph NOFD /proc/32179/task/84714/fd (opendir: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84715 m-fin-dev ceph cwd unknown /proc/32179/task/84715/cwd (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84715 m-fin-dev ceph rtd unknown /proc/32179/task/84715/root (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84715 m-fin-dev ceph txt unknown /proc/32179/task/84715/exe (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84715 m-fin-dev ceph NOFD /proc/32179/task/84715/fd (opendir: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84716 mgr-fin ceph cwd unknown /proc/32179/task/84716/cwd (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84716 mgr-fin ceph rtd unknown /proc/32179/task/84716/root (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84716 mgr-fin ceph txt unknown /proc/32179/task/84716/exe (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84716 mgr-fin ceph NOFD /proc/32179/task/84716/fd (opendir: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84718 m-fin-ios ceph cwd unknown /proc/32179/task/84718/cwd (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84718 m-fin-ios ceph rtd unknown /proc/32179/task/84718/root (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84718 m-fin-ios ceph txt unknown /proc/32179/task/84718/exe (readlink: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84718 m-fin-ios ceph NOFD /proc/32179/task/84718/fd (opendir: Permission denied) 2024-03-16T08:19:37.725 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84720 m-fin-nfs ceph cwd unknown /proc/32179/task/84720/cwd (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84720 m-fin-nfs ceph rtd unknown /proc/32179/task/84720/root (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84720 m-fin-nfs ceph txt unknown /proc/32179/task/84720/exe (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84720 m-fin-nfs ceph NOFD /proc/32179/task/84720/fd (opendir: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84722 m-fin-orc ceph cwd unknown /proc/32179/task/84722/cwd (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84722 m-fin-orc ceph rtd unknown /proc/32179/task/84722/root (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84722 m-fin-orc ceph txt unknown /proc/32179/task/84722/exe (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84722 m-fin-orc ceph NOFD /proc/32179/task/84722/fd (opendir: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84723 pg_autosc ceph cwd unknown /proc/32179/task/84723/cwd (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84723 pg_autosc ceph rtd unknown /proc/32179/task/84723/root (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84723 pg_autosc ceph txt unknown /proc/32179/task/84723/exe (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84723 pg_autosc ceph NOFD /proc/32179/task/84723/fd (opendir: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84724 m-fin-pg_ ceph cwd unknown /proc/32179/task/84724/cwd (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84724 m-fin-pg_ ceph rtd unknown /proc/32179/task/84724/root (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84724 m-fin-pg_ ceph txt unknown /proc/32179/task/84724/exe (readlink: Permission denied) 2024-03-16T08:19:37.726 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84724 m-fin-pg_ ceph NOFD /proc/32179/task/84724/fd (opendir: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84725 progress ceph cwd unknown /proc/32179/task/84725/cwd (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84725 progress ceph rtd unknown /proc/32179/task/84725/root (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84725 progress ceph txt unknown /proc/32179/task/84725/exe (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84725 progress ceph NOFD /proc/32179/task/84725/fd (opendir: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84726 m-fin-pro ceph cwd unknown /proc/32179/task/84726/cwd (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84726 m-fin-pro ceph rtd unknown /proc/32179/task/84726/root (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84726 m-fin-pro ceph txt unknown /proc/32179/task/84726/exe (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84726 m-fin-pro ceph NOFD /proc/32179/task/84726/fd (opendir: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84730 io_contex ceph cwd unknown /proc/32179/task/84730/cwd (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84730 io_contex ceph rtd unknown /proc/32179/task/84730/root (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84730 io_contex ceph txt unknown /proc/32179/task/84730/exe (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84730 io_contex ceph NOFD /proc/32179/task/84730/fd (opendir: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84731 io_contex ceph cwd unknown /proc/32179/task/84731/cwd (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84731 io_contex ceph rtd unknown /proc/32179/task/84731/root (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84731 io_contex ceph txt unknown /proc/32179/task/84731/exe (readlink: Permission denied) 2024-03-16T08:19:37.727 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84731 io_contex ceph NOFD /proc/32179/task/84731/fd (opendir: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84732 ceph_time ceph cwd unknown /proc/32179/task/84732/cwd (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84732 ceph_time ceph rtd unknown /proc/32179/task/84732/root (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84732 ceph_time ceph txt unknown /proc/32179/task/84732/exe (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84732 ceph_time ceph NOFD /proc/32179/task/84732/fd (opendir: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84733 ms_dispat ceph cwd unknown /proc/32179/task/84733/cwd (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84733 ms_dispat ceph rtd unknown /proc/32179/task/84733/root (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84733 ms_dispat ceph txt unknown /proc/32179/task/84733/exe (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84733 ms_dispat ceph NOFD /proc/32179/task/84733/fd (opendir: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84734 ms_local ceph cwd unknown /proc/32179/task/84734/cwd (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84734 ms_local ceph rtd unknown /proc/32179/task/84734/root (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84734 ms_local ceph txt unknown /proc/32179/task/84734/exe (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84734 ms_local ceph NOFD /proc/32179/task/84734/fd (opendir: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84735 safe_time ceph cwd unknown /proc/32179/task/84735/cwd (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84735 safe_time ceph rtd unknown /proc/32179/task/84735/root (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84735 safe_time ceph txt unknown /proc/32179/task/84735/exe (readlink: Permission denied) 2024-03-16T08:19:37.728 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84735 safe_time ceph NOFD /proc/32179/task/84735/fd (opendir: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84736 safe_time ceph cwd unknown /proc/32179/task/84736/cwd (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84736 safe_time ceph rtd unknown /proc/32179/task/84736/root (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84736 safe_time ceph txt unknown /proc/32179/task/84736/exe (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84736 safe_time ceph NOFD /proc/32179/task/84736/fd (opendir: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84737 mgr-fin ceph cwd unknown /proc/32179/task/84737/cwd (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84737 mgr-fin ceph rtd unknown /proc/32179/task/84737/root (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84737 mgr-fin ceph txt unknown /proc/32179/task/84737/exe (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84737 mgr-fin ceph NOFD /proc/32179/task/84737/fd (opendir: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84739 m-fin-rbd ceph cwd unknown /proc/32179/task/84739/cwd (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84739 m-fin-rbd ceph rtd unknown /proc/32179/task/84739/root (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84739 m-fin-rbd ceph txt unknown /proc/32179/task/84739/exe (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84739 m-fin-rbd ceph NOFD /proc/32179/task/84739/fd (opendir: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84740 restful ceph cwd unknown /proc/32179/task/84740/cwd (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84740 restful ceph rtd unknown /proc/32179/task/84740/root (readlink: Permission denied) 2024-03-16T08:19:37.729 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84740 restful ceph txt unknown /proc/32179/task/84740/exe (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84740 restful ceph NOFD /proc/32179/task/84740/fd (opendir: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84741 m-fin-res ceph cwd unknown /proc/32179/task/84741/cwd (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84741 m-fin-res ceph rtd unknown /proc/32179/task/84741/root (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84741 m-fin-res ceph txt unknown /proc/32179/task/84741/exe (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84741 m-fin-res ceph NOFD /proc/32179/task/84741/fd (opendir: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84742 mgr-fin ceph cwd unknown /proc/32179/task/84742/cwd (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84742 mgr-fin ceph rtd unknown /proc/32179/task/84742/root (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84742 mgr-fin ceph txt unknown /proc/32179/task/84742/exe (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84742 mgr-fin ceph NOFD /proc/32179/task/84742/fd (opendir: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84743 mgr-fin ceph cwd unknown /proc/32179/task/84743/cwd (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84743 mgr-fin ceph rtd unknown /proc/32179/task/84743/root (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84743 mgr-fin ceph txt unknown /proc/32179/task/84743/exe (readlink: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84743 mgr-fin ceph NOFD /proc/32179/task/84743/fd (opendir: Permission denied) 2024-03-16T08:19:37.730 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84744 mgr-fin ceph cwd unknown /proc/32179/task/84744/cwd (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84744 mgr-fin ceph rtd unknown /proc/32179/task/84744/root (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84744 mgr-fin ceph txt unknown /proc/32179/task/84744/exe (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84744 mgr-fin ceph NOFD /proc/32179/task/84744/fd (opendir: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84745 mgr-fin ceph cwd unknown /proc/32179/task/84745/cwd (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84745 mgr-fin ceph rtd unknown /proc/32179/task/84745/root (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84745 mgr-fin ceph txt unknown /proc/32179/task/84745/exe (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84745 mgr-fin ceph NOFD /proc/32179/task/84745/fd (opendir: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84746 mgr-fin ceph cwd unknown /proc/32179/task/84746/cwd (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84746 mgr-fin ceph rtd unknown /proc/32179/task/84746/root (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84746 mgr-fin ceph txt unknown /proc/32179/task/84746/exe (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84746 mgr-fin ceph NOFD /proc/32179/task/84746/fd (opendir: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84747 mgr-fin ceph cwd unknown /proc/32179/task/84747/cwd (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84747 mgr-fin ceph rtd unknown /proc/32179/task/84747/root (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84747 mgr-fin ceph txt unknown /proc/32179/task/84747/exe (readlink: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84747 mgr-fin ceph NOFD /proc/32179/task/84747/fd (opendir: Permission denied) 2024-03-16T08:19:37.731 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84751 io_contex ceph cwd unknown /proc/32179/task/84751/cwd (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84751 io_contex ceph rtd unknown /proc/32179/task/84751/root (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84751 io_contex ceph txt unknown /proc/32179/task/84751/exe (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84751 io_contex ceph NOFD /proc/32179/task/84751/fd (opendir: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84752 io_contex ceph cwd unknown /proc/32179/task/84752/cwd (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84752 io_contex ceph rtd unknown /proc/32179/task/84752/root (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84752 io_contex ceph txt unknown /proc/32179/task/84752/exe (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84752 io_contex ceph NOFD /proc/32179/task/84752/fd (opendir: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84753 ceph_time ceph cwd unknown /proc/32179/task/84753/cwd (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84753 ceph_time ceph rtd unknown /proc/32179/task/84753/root (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84753 ceph_time ceph txt unknown /proc/32179/task/84753/exe (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84753 ceph_time ceph NOFD /proc/32179/task/84753/fd (opendir: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84754 ms_dispat ceph cwd unknown /proc/32179/task/84754/cwd (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84754 ms_dispat ceph rtd unknown /proc/32179/task/84754/root (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84754 ms_dispat ceph txt unknown /proc/32179/task/84754/exe (readlink: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84754 ms_dispat ceph NOFD /proc/32179/task/84754/fd (opendir: Permission denied) 2024-03-16T08:19:37.732 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84755 ms_local ceph cwd unknown /proc/32179/task/84755/cwd (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84755 ms_local ceph rtd unknown /proc/32179/task/84755/root (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84755 ms_local ceph txt unknown /proc/32179/task/84755/exe (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84755 ms_local ceph NOFD /proc/32179/task/84755/fd (opendir: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84756 safe_time ceph cwd unknown /proc/32179/task/84756/cwd (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84756 safe_time ceph rtd unknown /proc/32179/task/84756/root (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84756 safe_time ceph txt unknown /proc/32179/task/84756/exe (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84756 safe_time ceph NOFD /proc/32179/task/84756/fd (opendir: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84757 safe_time ceph cwd unknown /proc/32179/task/84757/cwd (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84757 safe_time ceph rtd unknown /proc/32179/task/84757/root (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84757 safe_time ceph txt unknown /proc/32179/task/84757/exe (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84757 safe_time ceph NOFD /proc/32179/task/84757/fd (opendir: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84761 m-fin-sna ceph cwd unknown /proc/32179/task/84761/cwd (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84761 m-fin-sna ceph rtd unknown /proc/32179/task/84761/root (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84761 m-fin-sna ceph txt unknown /proc/32179/task/84761/exe (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84761 m-fin-sna ceph NOFD /proc/32179/task/84761/fd (opendir: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84763 m-fin-sta ceph cwd unknown /proc/32179/task/84763/cwd (readlink: Permission denied) 2024-03-16T08:19:37.733 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84763 m-fin-sta ceph rtd unknown /proc/32179/task/84763/root (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84763 m-fin-sta ceph txt unknown /proc/32179/task/84763/exe (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84763 m-fin-sta ceph NOFD /proc/32179/task/84763/fd (opendir: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84764 telemetry ceph cwd unknown /proc/32179/task/84764/cwd (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84764 telemetry ceph rtd unknown /proc/32179/task/84764/root (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84764 telemetry ceph txt unknown /proc/32179/task/84764/exe (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84764 telemetry ceph NOFD /proc/32179/task/84764/fd (opendir: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84765 m-fin-tel ceph cwd unknown /proc/32179/task/84765/cwd (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84765 m-fin-tel ceph rtd unknown /proc/32179/task/84765/root (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84765 m-fin-tel ceph txt unknown /proc/32179/task/84765/exe (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84765 m-fin-tel ceph NOFD /proc/32179/task/84765/fd (opendir: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84766 mgr-fin ceph cwd unknown /proc/32179/task/84766/cwd (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84766 mgr-fin ceph rtd unknown /proc/32179/task/84766/root (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84766 mgr-fin ceph txt unknown /proc/32179/task/84766/exe (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84766 mgr-fin ceph NOFD /proc/32179/task/84766/fd (opendir: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84770 io_contex ceph cwd unknown /proc/32179/task/84770/cwd (readlink: Permission denied) 2024-03-16T08:19:37.734 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84770 io_contex ceph rtd unknown /proc/32179/task/84770/root (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84770 io_contex ceph txt unknown /proc/32179/task/84770/exe (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84770 io_contex ceph NOFD /proc/32179/task/84770/fd (opendir: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84771 io_contex ceph cwd unknown /proc/32179/task/84771/cwd (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84771 io_contex ceph rtd unknown /proc/32179/task/84771/root (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84771 io_contex ceph txt unknown /proc/32179/task/84771/exe (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84771 io_contex ceph NOFD /proc/32179/task/84771/fd (opendir: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84772 ceph_time ceph cwd unknown /proc/32179/task/84772/cwd (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84772 ceph_time ceph rtd unknown /proc/32179/task/84772/root (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84772 ceph_time ceph txt unknown /proc/32179/task/84772/exe (readlink: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84772 ceph_time ceph NOFD /proc/32179/task/84772/fd (opendir: Permission denied) 2024-03-16T08:19:37.735 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84773 ms_dispat ceph cwd unknown /proc/32179/task/84773/cwd (readlink: Permission denied) 2024-03-16T08:19:37.737 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84773 ms_dispat ceph rtd unknown /proc/32179/task/84773/root (readlink: Permission denied) 2024-03-16T08:19:37.737 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84773 ms_dispat ceph txt unknown /proc/32179/task/84773/exe (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84773 ms_dispat ceph NOFD /proc/32179/task/84773/fd (opendir: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84774 ms_local ceph cwd unknown /proc/32179/task/84774/cwd (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84774 ms_local ceph rtd unknown /proc/32179/task/84774/root (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84774 ms_local ceph txt unknown /proc/32179/task/84774/exe (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84774 ms_local ceph NOFD /proc/32179/task/84774/fd (opendir: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84775 safe_time ceph cwd unknown /proc/32179/task/84775/cwd (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84775 safe_time ceph rtd unknown /proc/32179/task/84775/root (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84775 safe_time ceph txt unknown /proc/32179/task/84775/exe (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84775 safe_time ceph NOFD /proc/32179/task/84775/fd (opendir: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84776 safe_time ceph cwd unknown /proc/32179/task/84776/cwd (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84776 safe_time ceph rtd unknown /proc/32179/task/84776/root (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84776 safe_time ceph txt unknown /proc/32179/task/84776/exe (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84776 safe_time ceph NOFD /proc/32179/task/84776/fd (opendir: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84777 mgr-fin ceph cwd unknown /proc/32179/task/84777/cwd (readlink: Permission denied) 2024-03-16T08:19:37.738 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84777 mgr-fin ceph rtd unknown /proc/32179/task/84777/root (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84777 mgr-fin ceph txt unknown /proc/32179/task/84777/exe (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84777 mgr-fin ceph NOFD /proc/32179/task/84777/fd (opendir: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84778 mgr-fin ceph cwd unknown /proc/32179/task/84778/cwd (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84778 mgr-fin ceph rtd unknown /proc/32179/task/84778/root (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84778 mgr-fin ceph txt unknown /proc/32179/task/84778/exe (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84778 mgr-fin ceph NOFD /proc/32179/task/84778/fd (opendir: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84779 mgr-fin ceph cwd unknown /proc/32179/task/84779/cwd (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84779 mgr-fin ceph rtd unknown /proc/32179/task/84779/root (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84779 mgr-fin ceph txt unknown /proc/32179/task/84779/exe (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84779 mgr-fin ceph NOFD /proc/32179/task/84779/fd (opendir: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84780 mgr-fin ceph cwd unknown /proc/32179/task/84780/cwd (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84780 mgr-fin ceph rtd unknown /proc/32179/task/84780/root (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84780 mgr-fin ceph txt unknown /proc/32179/task/84780/exe (readlink: Permission denied) 2024-03-16T08:19:37.739 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84780 mgr-fin ceph NOFD /proc/32179/task/84780/fd (opendir: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84781 mgr-fin ceph cwd unknown /proc/32179/task/84781/cwd (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84781 mgr-fin ceph rtd unknown /proc/32179/task/84781/root (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84781 mgr-fin ceph txt unknown /proc/32179/task/84781/exe (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84781 mgr-fin ceph NOFD /proc/32179/task/84781/fd (opendir: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84782 mgr-fin ceph cwd unknown /proc/32179/task/84782/cwd (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84782 mgr-fin ceph rtd unknown /proc/32179/task/84782/root (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84782 mgr-fin ceph txt unknown /proc/32179/task/84782/exe (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84782 mgr-fin ceph NOFD /proc/32179/task/84782/fd (opendir: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84783 mgr-fin ceph cwd unknown /proc/32179/task/84783/cwd (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84783 mgr-fin ceph rtd unknown /proc/32179/task/84783/root (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84783 mgr-fin ceph txt unknown /proc/32179/task/84783/exe (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84783 mgr-fin ceph NOFD /proc/32179/task/84783/fd (opendir: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84784 mgr-fin ceph cwd unknown /proc/32179/task/84784/cwd (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84784 mgr-fin ceph rtd unknown /proc/32179/task/84784/root (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84784 mgr-fin ceph txt unknown /proc/32179/task/84784/exe (readlink: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84784 mgr-fin ceph NOFD /proc/32179/task/84784/fd (opendir: Permission denied) 2024-03-16T08:19:37.740 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84785 mgr-fin ceph cwd unknown /proc/32179/task/84785/cwd (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84785 mgr-fin ceph rtd unknown /proc/32179/task/84785/root (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84785 mgr-fin ceph txt unknown /proc/32179/task/84785/exe (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84785 mgr-fin ceph NOFD /proc/32179/task/84785/fd (opendir: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84786 mgr-fin ceph cwd unknown /proc/32179/task/84786/cwd (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84786 mgr-fin ceph rtd unknown /proc/32179/task/84786/root (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84786 mgr-fin ceph txt unknown /proc/32179/task/84786/exe (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84786 mgr-fin ceph NOFD /proc/32179/task/84786/fd (opendir: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84787 mgr-fin ceph cwd unknown /proc/32179/task/84787/cwd (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84787 mgr-fin ceph rtd unknown /proc/32179/task/84787/root (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84787 mgr-fin ceph txt unknown /proc/32179/task/84787/exe (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84787 mgr-fin ceph NOFD /proc/32179/task/84787/fd (opendir: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84788 mgr-fin ceph cwd unknown /proc/32179/task/84788/cwd (readlink: Permission denied) 2024-03-16T08:19:37.741 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84788 mgr-fin ceph rtd unknown /proc/32179/task/84788/root (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84788 mgr-fin ceph txt unknown /proc/32179/task/84788/exe (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84788 mgr-fin ceph NOFD /proc/32179/task/84788/fd (opendir: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84790 m-fin-vol ceph cwd unknown /proc/32179/task/84790/cwd (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84790 m-fin-vol ceph rtd unknown /proc/32179/task/84790/root (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84790 m-fin-vol ceph txt unknown /proc/32179/task/84790/exe (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84790 m-fin-vol ceph NOFD /proc/32179/task/84790/fd (opendir: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84791 io_contex ceph cwd unknown /proc/32179/task/84791/cwd (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84791 io_contex ceph rtd unknown /proc/32179/task/84791/root (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84791 io_contex ceph txt unknown /proc/32179/task/84791/exe (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84791 io_contex ceph NOFD /proc/32179/task/84791/fd (opendir: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84792 io_contex ceph cwd unknown /proc/32179/task/84792/cwd (readlink: Permission denied) 2024-03-16T08:19:37.742 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84792 io_contex ceph rtd unknown /proc/32179/task/84792/root (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84792 io_contex ceph txt unknown /proc/32179/task/84792/exe (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84792 io_contex ceph NOFD /proc/32179/task/84792/fd (opendir: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84824 dashboard ceph cwd unknown /proc/32179/task/84824/cwd (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84824 dashboard ceph rtd unknown /proc/32179/task/84824/root (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84824 dashboard ceph txt unknown /proc/32179/task/84824/exe (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84824 dashboard ceph NOFD /proc/32179/task/84824/fd (opendir: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84826 dashboard ceph cwd unknown /proc/32179/task/84826/cwd (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84826 dashboard ceph rtd unknown /proc/32179/task/84826/root (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84826 dashboard ceph txt unknown /proc/32179/task/84826/exe (readlink: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84826 dashboard ceph NOFD /proc/32179/task/84826/fd (opendir: Permission denied) 2024-03-16T08:19:37.743 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84828 dashboard ceph cwd unknown /proc/32179/task/84828/cwd (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84828 dashboard ceph rtd unknown /proc/32179/task/84828/root (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84828 dashboard ceph txt unknown /proc/32179/task/84828/exe (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84828 dashboard ceph NOFD /proc/32179/task/84828/fd (opendir: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84829 dashboard ceph cwd unknown /proc/32179/task/84829/cwd (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84829 dashboard ceph rtd unknown /proc/32179/task/84829/root (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84829 dashboard ceph txt unknown /proc/32179/task/84829/exe (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84829 dashboard ceph NOFD /proc/32179/task/84829/fd (opendir: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84830 dashboard ceph cwd unknown /proc/32179/task/84830/cwd (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84830 dashboard ceph rtd unknown /proc/32179/task/84830/root (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84830 dashboard ceph txt unknown /proc/32179/task/84830/exe (readlink: Permission denied) 2024-03-16T08:19:37.744 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84830 dashboard ceph NOFD /proc/32179/task/84830/fd (opendir: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84832 dashboard ceph cwd unknown /proc/32179/task/84832/cwd (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84832 dashboard ceph rtd unknown /proc/32179/task/84832/root (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84832 dashboard ceph txt unknown /proc/32179/task/84832/exe (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84832 dashboard ceph NOFD /proc/32179/task/84832/fd (opendir: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84833 dashboard ceph cwd unknown /proc/32179/task/84833/cwd (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84833 dashboard ceph rtd unknown /proc/32179/task/84833/root (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84833 dashboard ceph txt unknown /proc/32179/task/84833/exe (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84833 dashboard ceph NOFD /proc/32179/task/84833/fd (opendir: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84834 dashboard ceph cwd unknown /proc/32179/task/84834/cwd (readlink: Permission denied) 2024-03-16T08:19:37.745 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84834 dashboard ceph rtd unknown /proc/32179/task/84834/root (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84834 dashboard ceph txt unknown /proc/32179/task/84834/exe (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84834 dashboard ceph NOFD /proc/32179/task/84834/fd (opendir: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84835 dashboard ceph cwd unknown /proc/32179/task/84835/cwd (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84835 dashboard ceph rtd unknown /proc/32179/task/84835/root (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84835 dashboard ceph txt unknown /proc/32179/task/84835/exe (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84835 dashboard ceph NOFD /proc/32179/task/84835/fd (opendir: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84837 dashboard ceph cwd unknown /proc/32179/task/84837/cwd (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84837 dashboard ceph rtd unknown /proc/32179/task/84837/root (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84837 dashboard ceph txt unknown /proc/32179/task/84837/exe (readlink: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84837 dashboard ceph NOFD /proc/32179/task/84837/fd (opendir: Permission denied) 2024-03-16T08:19:37.746 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84838 dashboard ceph cwd unknown /proc/32179/task/84838/cwd (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84838 dashboard ceph rtd unknown /proc/32179/task/84838/root (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84838 dashboard ceph txt unknown /proc/32179/task/84838/exe (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84838 dashboard ceph NOFD /proc/32179/task/84838/fd (opendir: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84873 mgr-fin ceph cwd unknown /proc/32179/task/84873/cwd (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84873 mgr-fin ceph rtd unknown /proc/32179/task/84873/root (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84873 mgr-fin ceph txt unknown /proc/32179/task/84873/exe (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84873 mgr-fin ceph NOFD /proc/32179/task/84873/fd (opendir: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84875 mgr-fin ceph cwd unknown /proc/32179/task/84875/cwd (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84875 mgr-fin ceph rtd unknown /proc/32179/task/84875/root (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84875 mgr-fin ceph txt unknown /proc/32179/task/84875/exe (readlink: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84875 mgr-fin ceph NOFD /proc/32179/task/84875/fd (opendir: Permission denied) 2024-03-16T08:19:37.747 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84876 mgr-fin ceph cwd unknown /proc/32179/task/84876/cwd (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84876 mgr-fin ceph rtd unknown /proc/32179/task/84876/root (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84876 mgr-fin ceph txt unknown /proc/32179/task/84876/exe (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84876 mgr-fin ceph NOFD /proc/32179/task/84876/fd (opendir: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84877 mgr-fin ceph cwd unknown /proc/32179/task/84877/cwd (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84877 mgr-fin ceph rtd unknown /proc/32179/task/84877/root (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84877 mgr-fin ceph txt unknown /proc/32179/task/84877/exe (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84877 mgr-fin ceph NOFD /proc/32179/task/84877/fd (opendir: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84878 mgr-fin ceph cwd unknown /proc/32179/task/84878/cwd (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84878 mgr-fin ceph rtd unknown /proc/32179/task/84878/root (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84878 mgr-fin ceph txt unknown /proc/32179/task/84878/exe (readlink: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84878 mgr-fin ceph NOFD /proc/32179/task/84878/fd (opendir: Permission denied) 2024-03-16T08:19:37.748 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84879 mgr-fin ceph cwd unknown /proc/32179/task/84879/cwd (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84879 mgr-fin ceph rtd unknown /proc/32179/task/84879/root (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84879 mgr-fin ceph txt unknown /proc/32179/task/84879/exe (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84879 mgr-fin ceph NOFD /proc/32179/task/84879/fd (opendir: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84882 mgr-fin ceph cwd unknown /proc/32179/task/84882/cwd (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84882 mgr-fin ceph rtd unknown /proc/32179/task/84882/root (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84882 mgr-fin ceph txt unknown /proc/32179/task/84882/exe (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84882 mgr-fin ceph NOFD /proc/32179/task/84882/fd (opendir: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84884 mgr-fin ceph cwd unknown /proc/32179/task/84884/cwd (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84884 mgr-fin ceph rtd unknown /proc/32179/task/84884/root (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84884 mgr-fin ceph txt unknown /proc/32179/task/84884/exe (readlink: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84884 mgr-fin ceph NOFD /proc/32179/task/84884/fd (opendir: Permission denied) 2024-03-16T08:19:37.749 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84885 mgr-fin ceph cwd unknown /proc/32179/task/84885/cwd (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84885 mgr-fin ceph rtd unknown /proc/32179/task/84885/root (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84885 mgr-fin ceph txt unknown /proc/32179/task/84885/exe (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84885 mgr-fin ceph NOFD /proc/32179/task/84885/fd (opendir: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84886 mgr-fin ceph cwd unknown /proc/32179/task/84886/cwd (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84886 mgr-fin ceph rtd unknown /proc/32179/task/84886/root (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84886 mgr-fin ceph txt unknown /proc/32179/task/84886/exe (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84886 mgr-fin ceph NOFD /proc/32179/task/84886/fd (opendir: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84888 mgr-fin ceph cwd unknown /proc/32179/task/84888/cwd (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84888 mgr-fin ceph rtd unknown /proc/32179/task/84888/root (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84888 mgr-fin ceph txt unknown /proc/32179/task/84888/exe (readlink: Permission denied) 2024-03-16T08:19:37.750 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84888 mgr-fin ceph NOFD /proc/32179/task/84888/fd (opendir: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84909 dashboard ceph cwd unknown /proc/32179/task/84909/cwd (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84909 dashboard ceph rtd unknown /proc/32179/task/84909/root (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84909 dashboard ceph txt unknown /proc/32179/task/84909/exe (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84909 dashboard ceph NOFD /proc/32179/task/84909/fd (opendir: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84911 mgr-fin ceph cwd unknown /proc/32179/task/84911/cwd (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84911 mgr-fin ceph rtd unknown /proc/32179/task/84911/root (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84911 mgr-fin ceph txt unknown /proc/32179/task/84911/exe (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84911 mgr-fin ceph NOFD /proc/32179/task/84911/fd (opendir: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84912 mgr-fin ceph cwd unknown /proc/32179/task/84912/cwd (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84912 mgr-fin ceph rtd unknown /proc/32179/task/84912/root (readlink: Permission denied) 2024-03-16T08:19:37.751 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84912 mgr-fin ceph txt unknown /proc/32179/task/84912/exe (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84912 mgr-fin ceph NOFD /proc/32179/task/84912/fd (opendir: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84913 mgr-fin ceph cwd unknown /proc/32179/task/84913/cwd (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84913 mgr-fin ceph rtd unknown /proc/32179/task/84913/root (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84913 mgr-fin ceph txt unknown /proc/32179/task/84913/exe (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84913 mgr-fin ceph NOFD /proc/32179/task/84913/fd (opendir: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84914 mgr-fin ceph cwd unknown /proc/32179/task/84914/cwd (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84914 mgr-fin ceph rtd unknown /proc/32179/task/84914/root (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84914 mgr-fin ceph txt unknown /proc/32179/task/84914/exe (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84914 mgr-fin ceph NOFD /proc/32179/task/84914/fd (opendir: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84915 mgr-fin ceph cwd unknown /proc/32179/task/84915/cwd (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84915 mgr-fin ceph rtd unknown /proc/32179/task/84915/root (readlink: Permission denied) 2024-03-16T08:19:37.752 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84915 mgr-fin ceph txt unknown /proc/32179/task/84915/exe (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84915 mgr-fin ceph NOFD /proc/32179/task/84915/fd (opendir: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84916 mgr-fin ceph cwd unknown /proc/32179/task/84916/cwd (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84916 mgr-fin ceph rtd unknown /proc/32179/task/84916/root (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84916 mgr-fin ceph txt unknown /proc/32179/task/84916/exe (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84916 mgr-fin ceph NOFD /proc/32179/task/84916/fd (opendir: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84917 mgr-fin ceph cwd unknown /proc/32179/task/84917/cwd (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84917 mgr-fin ceph rtd unknown /proc/32179/task/84917/root (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84917 mgr-fin ceph txt unknown /proc/32179/task/84917/exe (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84917 mgr-fin ceph NOFD /proc/32179/task/84917/fd (opendir: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84918 mgr-fin ceph cwd unknown /proc/32179/task/84918/cwd (readlink: Permission denied) 2024-03-16T08:19:37.753 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84918 mgr-fin ceph rtd unknown /proc/32179/task/84918/root (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84918 mgr-fin ceph txt unknown /proc/32179/task/84918/exe (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84918 mgr-fin ceph NOFD /proc/32179/task/84918/fd (opendir: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84919 mgr-fin ceph cwd unknown /proc/32179/task/84919/cwd (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84919 mgr-fin ceph rtd unknown /proc/32179/task/84919/root (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84919 mgr-fin ceph txt unknown /proc/32179/task/84919/exe (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84919 mgr-fin ceph NOFD /proc/32179/task/84919/fd (opendir: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84920 mgr-fin ceph cwd unknown /proc/32179/task/84920/cwd (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84920 mgr-fin ceph rtd unknown /proc/32179/task/84920/root (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84920 mgr-fin ceph txt unknown /proc/32179/task/84920/exe (readlink: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84920 mgr-fin ceph NOFD /proc/32179/task/84920/fd (opendir: Permission denied) 2024-03-16T08:19:37.754 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84921 mgr-fin ceph cwd unknown /proc/32179/task/84921/cwd (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84921 mgr-fin ceph rtd unknown /proc/32179/task/84921/root (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84921 mgr-fin ceph txt unknown /proc/32179/task/84921/exe (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 84921 mgr-fin ceph NOFD /proc/32179/task/84921/fd (opendir: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85301 devicehea ceph cwd unknown /proc/32179/task/85301/cwd (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85301 devicehea ceph rtd unknown /proc/32179/task/85301/root (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85301 devicehea ceph txt unknown /proc/32179/task/85301/exe (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85301 devicehea ceph NOFD /proc/32179/task/85301/fd (opendir: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85317 io_contex ceph cwd unknown /proc/32179/task/85317/cwd (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85317 io_contex ceph rtd unknown /proc/32179/task/85317/root (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85317 io_contex ceph txt unknown /proc/32179/task/85317/exe (readlink: Permission denied) 2024-03-16T08:19:37.755 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85317 io_contex ceph NOFD /proc/32179/task/85317/fd (opendir: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85318 io_contex ceph cwd unknown /proc/32179/task/85318/cwd (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85318 io_contex ceph rtd unknown /proc/32179/task/85318/root (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85318 io_contex ceph txt unknown /proc/32179/task/85318/exe (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85318 io_contex ceph NOFD /proc/32179/task/85318/fd (opendir: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85319 ceph_time ceph cwd unknown /proc/32179/task/85319/cwd (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85319 ceph_time ceph rtd unknown /proc/32179/task/85319/root (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85319 ceph_time ceph txt unknown /proc/32179/task/85319/exe (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85319 ceph_time ceph NOFD /proc/32179/task/85319/fd (opendir: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85320 ms_dispat ceph cwd unknown /proc/32179/task/85320/cwd (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85320 ms_dispat ceph rtd unknown /proc/32179/task/85320/root (readlink: Permission denied) 2024-03-16T08:19:37.756 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85320 ms_dispat ceph txt unknown /proc/32179/task/85320/exe (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85320 ms_dispat ceph NOFD /proc/32179/task/85320/fd (opendir: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85321 ms_local ceph cwd unknown /proc/32179/task/85321/cwd (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85321 ms_local ceph rtd unknown /proc/32179/task/85321/root (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85321 ms_local ceph txt unknown /proc/32179/task/85321/exe (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85321 ms_local ceph NOFD /proc/32179/task/85321/fd (opendir: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85322 safe_time ceph cwd unknown /proc/32179/task/85322/cwd (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85322 safe_time ceph rtd unknown /proc/32179/task/85322/root (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85322 safe_time ceph txt unknown /proc/32179/task/85322/exe (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85322 safe_time ceph NOFD /proc/32179/task/85322/fd (opendir: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85323 safe_time ceph cwd unknown /proc/32179/task/85323/cwd (readlink: Permission denied) 2024-03-16T08:19:37.757 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85323 safe_time ceph rtd unknown /proc/32179/task/85323/root (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85323 safe_time ceph txt unknown /proc/32179/task/85323/exe (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 85323 safe_time ceph NOFD /proc/32179/task/85323/fd (opendir: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87455 ceph_time ceph cwd unknown /proc/32179/task/87455/cwd (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87455 ceph_time ceph rtd unknown /proc/32179/task/87455/root (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87455 ceph_time ceph txt unknown /proc/32179/task/87455/exe (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87455 ceph_time ceph NOFD /proc/32179/task/87455/fd (opendir: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87456 fn_anonym ceph cwd unknown /proc/32179/task/87456/cwd (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87456 fn_anonym ceph rtd unknown /proc/32179/task/87456/root (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87456 fn_anonym ceph txt unknown /proc/32179/task/87456/exe (readlink: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87456 fn_anonym ceph NOFD /proc/32179/task/87456/fd (opendir: Permission denied) 2024-03-16T08:19:37.758 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87457 safe_time ceph cwd unknown /proc/32179/task/87457/cwd (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87457 safe_time ceph rtd unknown /proc/32179/task/87457/root (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87457 safe_time ceph txt unknown /proc/32179/task/87457/exe (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87457 safe_time ceph NOFD /proc/32179/task/87457/fd (opendir: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87458 fn_anonym ceph cwd unknown /proc/32179/task/87458/cwd (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87458 fn_anonym ceph rtd unknown /proc/32179/task/87458/root (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87458 fn_anonym ceph txt unknown /proc/32179/task/87458/exe (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87458 fn_anonym ceph NOFD /proc/32179/task/87458/fd (opendir: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87459 flusher ceph cwd unknown /proc/32179/task/87459/cwd (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87459 flusher ceph rtd unknown /proc/32179/task/87459/root (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87459 flusher ceph txt unknown /proc/32179/task/87459/exe (readlink: Permission denied) 2024-03-16T08:19:37.759 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87459 flusher ceph NOFD /proc/32179/task/87459/fd (opendir: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87460 ms_dispat ceph cwd unknown /proc/32179/task/87460/cwd (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87460 ms_dispat ceph rtd unknown /proc/32179/task/87460/root (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87460 ms_dispat ceph txt unknown /proc/32179/task/87460/exe (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87460 ms_dispat ceph NOFD /proc/32179/task/87460/fd (opendir: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87461 ms_local ceph cwd unknown /proc/32179/task/87461/cwd (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87461 ms_local ceph rtd unknown /proc/32179/task/87461/root (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87461 ms_local ceph txt unknown /proc/32179/task/87461/exe (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87461 ms_local ceph NOFD /proc/32179/task/87461/fd (opendir: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87462 safe_time ceph cwd unknown /proc/32179/task/87462/cwd (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87462 safe_time ceph rtd unknown /proc/32179/task/87462/root (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87462 safe_time ceph txt unknown /proc/32179/task/87462/exe (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87462 safe_time ceph NOFD /proc/32179/task/87462/fd (opendir: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87463 m-fin-sna ceph cwd unknown /proc/32179/task/87463/cwd (readlink: Permission denied) 2024-03-16T08:19:37.760 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87463 m-fin-sna ceph rtd unknown /proc/32179/task/87463/root (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87463 m-fin-sna ceph txt unknown /proc/32179/task/87463/exe (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 87463 m-fin-sna ceph NOFD /proc/32179/task/87463/fd (opendir: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 88918 m-fin-sna ceph cwd unknown /proc/32179/task/88918/cwd (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 88918 m-fin-sna ceph rtd unknown /proc/32179/task/88918/root (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 88918 m-fin-sna ceph txt unknown /proc/32179/task/88918/exe (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mgr 32179 88918 m-fin-sna ceph NOFD /proc/32179/task/88918/fd (opendir: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 32285 root cwd unknown /proc/32285/cwd (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 32285 root rtd unknown /proc/32285/root (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 32285 root txt unknown /proc/32285/exe (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 32285 root NOFD /proc/32285/fd (opendir: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 33330 root cwd unknown /proc/33330/cwd (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 33330 root rtd unknown /proc/33330/root (readlink: Permission denied) 2024-03-16T08:19:37.761 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 33330 root txt unknown /proc/33330/exe (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:systemd 33330 root NOFD /proc/33330/fd (opendir: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 33333 root cwd unknown /proc/33333/cwd (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 33333 root rtd unknown /proc/33333/root (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 33333 root txt unknown /proc/33333/exe (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:(sd-pam) 33333 root NOFD /proc/33333/fd (opendir: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 46677 root cwd unknown /proc/46677/cwd (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 46677 root rtd unknown /proc/46677/root (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 46677 root txt unknown /proc/46677/exe (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 46677 root NOFD /proc/46677/fd (opendir: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 46679 root cwd unknown /proc/46679/cwd (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 46679 root rtd unknown /proc/46679/root (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 46679 root txt unknown /proc/46679/exe (readlink: Permission denied) 2024-03-16T08:19:37.762 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 46679 root NOFD /proc/46679/fd (opendir: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 ceph cwd unknown /proc/46682/cwd (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 ceph rtd unknown /proc/46682/root (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 ceph txt unknown /proc/46682/exe (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 ceph NOFD /proc/46682/fd (opendir: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46683 log ceph cwd unknown /proc/46682/task/46683/cwd (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46683 log ceph rtd unknown /proc/46682/task/46683/root (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46683 log ceph txt unknown /proc/46682/task/46683/exe (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46683 log ceph NOFD /proc/46682/task/46683/fd (opendir: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46685 msgr-work ceph cwd unknown /proc/46682/task/46685/cwd (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46685 msgr-work ceph rtd unknown /proc/46682/task/46685/root (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46685 msgr-work ceph txt unknown /proc/46682/task/46685/exe (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46685 msgr-work ceph NOFD /proc/46682/task/46685/fd (opendir: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46686 msgr-work ceph cwd unknown /proc/46682/task/46686/cwd (readlink: Permission denied) 2024-03-16T08:19:37.763 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46686 msgr-work ceph rtd unknown /proc/46682/task/46686/root (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46686 msgr-work ceph txt unknown /proc/46682/task/46686/exe (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46686 msgr-work ceph NOFD /proc/46682/task/46686/fd (opendir: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46687 msgr-work ceph cwd unknown /proc/46682/task/46687/cwd (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46687 msgr-work ceph rtd unknown /proc/46682/task/46687/root (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46687 msgr-work ceph txt unknown /proc/46682/task/46687/exe (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46687 msgr-work ceph NOFD /proc/46682/task/46687/fd (opendir: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46691 service ceph cwd unknown /proc/46682/task/46691/cwd (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46691 service ceph rtd unknown /proc/46682/task/46691/root (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46691 service ceph txt unknown /proc/46682/task/46691/exe (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46691 service ceph NOFD /proc/46682/task/46691/fd (opendir: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46692 admin_soc ceph cwd unknown /proc/46682/task/46692/cwd (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46692 admin_soc ceph rtd unknown /proc/46682/task/46692/root (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46692 admin_soc ceph txt unknown /proc/46682/task/46692/exe (readlink: Permission denied) 2024-03-16T08:19:37.764 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46692 admin_soc ceph NOFD /proc/46682/task/46692/fd (opendir: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46834 signal_ha ceph cwd unknown /proc/46682/task/46834/cwd (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46834 signal_ha ceph rtd unknown /proc/46682/task/46834/root (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46834 signal_ha ceph txt unknown /proc/46682/task/46834/exe (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46834 signal_ha ceph NOFD /proc/46682/task/46834/fd (opendir: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46835 io_contex ceph cwd unknown /proc/46682/task/46835/cwd (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46835 io_contex ceph rtd unknown /proc/46682/task/46835/root (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46835 io_contex ceph txt unknown /proc/46682/task/46835/exe (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46835 io_contex ceph NOFD /proc/46682/task/46835/fd (opendir: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46836 io_contex ceph cwd unknown /proc/46682/task/46836/cwd (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46836 io_contex ceph rtd unknown /proc/46682/task/46836/root (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46836 io_contex ceph txt unknown /proc/46682/task/46836/exe (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46836 io_contex ceph NOFD /proc/46682/task/46836/fd (opendir: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46954 OpHistory ceph cwd unknown /proc/46682/task/46954/cwd (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46954 OpHistory ceph rtd unknown /proc/46682/task/46954/root (readlink: Permission denied) 2024-03-16T08:19:37.765 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46954 OpHistory ceph txt unknown /proc/46682/task/46954/exe (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46954 OpHistory ceph NOFD /proc/46682/task/46954/fd (opendir: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46955 ceph_time ceph cwd unknown /proc/46682/task/46955/cwd (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46955 ceph_time ceph rtd unknown /proc/46682/task/46955/root (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46955 ceph_time ceph txt unknown /proc/46682/task/46955/exe (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 46955 ceph_time ceph NOFD /proc/46682/task/46955/fd (opendir: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47175 ceph-osd ceph cwd unknown /proc/46682/task/47175/cwd (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47175 ceph-osd ceph rtd unknown /proc/46682/task/47175/root (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47175 ceph-osd ceph txt unknown /proc/46682/task/47175/exe (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47175 ceph-osd ceph NOFD /proc/46682/task/47175/fd (opendir: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47176 safe_time ceph cwd unknown /proc/46682/task/47176/cwd (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47176 safe_time ceph rtd unknown /proc/46682/task/47176/root (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47176 safe_time ceph txt unknown /proc/46682/task/47176/exe (readlink: Permission denied) 2024-03-16T08:19:37.766 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47176 safe_time ceph NOFD /proc/46682/task/47176/fd (opendir: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47177 safe_time ceph cwd unknown /proc/46682/task/47177/cwd (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47177 safe_time ceph rtd unknown /proc/46682/task/47177/root (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47177 safe_time ceph txt unknown /proc/46682/task/47177/exe (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47177 safe_time ceph NOFD /proc/46682/task/47177/fd (opendir: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47178 safe_time ceph cwd unknown /proc/46682/task/47178/cwd (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47178 safe_time ceph rtd unknown /proc/46682/task/47178/root (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47178 safe_time ceph txt unknown /proc/46682/task/47178/exe (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47178 safe_time ceph NOFD /proc/46682/task/47178/fd (opendir: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47179 safe_time ceph cwd unknown /proc/46682/task/47179/cwd (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47179 safe_time ceph rtd unknown /proc/46682/task/47179/root (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47179 safe_time ceph txt unknown /proc/46682/task/47179/exe (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47179 safe_time ceph NOFD /proc/46682/task/47179/fd (opendir: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47180 fn_anonym ceph cwd unknown /proc/46682/task/47180/cwd (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47180 fn_anonym ceph rtd unknown /proc/46682/task/47180/root (readlink: Permission denied) 2024-03-16T08:19:37.767 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47180 fn_anonym ceph txt unknown /proc/46682/task/47180/exe (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47180 fn_anonym ceph NOFD /proc/46682/task/47180/fd (opendir: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47183 rocksdb:l ceph cwd unknown /proc/46682/task/47183/cwd (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47183 rocksdb:l ceph rtd unknown /proc/46682/task/47183/root (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47183 rocksdb:l ceph txt unknown /proc/46682/task/47183/exe (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47183 rocksdb:l ceph NOFD /proc/46682/task/47183/fd (opendir: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47184 rocksdb:l ceph cwd unknown /proc/46682/task/47184/cwd (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47184 rocksdb:l ceph rtd unknown /proc/46682/task/47184/root (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47184 rocksdb:l ceph txt unknown /proc/46682/task/47184/exe (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47184 rocksdb:l ceph NOFD /proc/46682/task/47184/fd (opendir: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47185 rocksdb:l ceph cwd unknown /proc/46682/task/47185/cwd (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47185 rocksdb:l ceph rtd unknown /proc/46682/task/47185/root (readlink: Permission denied) 2024-03-16T08:19:37.768 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47185 rocksdb:l ceph txt unknown /proc/46682/task/47185/exe (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47185 rocksdb:l ceph NOFD /proc/46682/task/47185/fd (opendir: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47186 rocksdb:h ceph cwd unknown /proc/46682/task/47186/cwd (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47186 rocksdb:h ceph rtd unknown /proc/46682/task/47186/root (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47186 rocksdb:h ceph txt unknown /proc/46682/task/47186/exe (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47186 rocksdb:h ceph NOFD /proc/46682/task/47186/fd (opendir: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47652 bstore_ai ceph cwd unknown /proc/46682/task/47652/cwd (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47652 bstore_ai ceph rtd unknown /proc/46682/task/47652/root (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47652 bstore_ai ceph txt unknown /proc/46682/task/47652/exe (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47652 bstore_ai ceph NOFD /proc/46682/task/47652/fd (opendir: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47837 bstore_ai ceph cwd unknown /proc/46682/task/47837/cwd (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47837 bstore_ai ceph rtd unknown /proc/46682/task/47837/root (readlink: Permission denied) 2024-03-16T08:19:37.769 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47837 bstore_ai ceph txt unknown /proc/46682/task/47837/exe (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 47837 bstore_ai ceph NOFD /proc/46682/task/47837/fd (opendir: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48019 ceph-osd ceph cwd unknown /proc/46682/task/48019/cwd (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48019 ceph-osd ceph rtd unknown /proc/46682/task/48019/root (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48019 ceph-osd ceph txt unknown /proc/46682/task/48019/exe (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48019 ceph-osd ceph NOFD /proc/46682/task/48019/fd (opendir: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48020 cfin ceph cwd unknown /proc/46682/task/48020/cwd (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48020 cfin ceph rtd unknown /proc/46682/task/48020/root (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48020 cfin ceph txt unknown /proc/46682/task/48020/exe (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48020 cfin ceph NOFD /proc/46682/task/48020/fd (opendir: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48021 bstore_kv ceph cwd unknown /proc/46682/task/48021/cwd (readlink: Permission denied) 2024-03-16T08:19:37.770 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48021 bstore_kv ceph rtd unknown /proc/46682/task/48021/root (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48021 bstore_kv ceph txt unknown /proc/46682/task/48021/exe (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48021 bstore_kv ceph NOFD /proc/46682/task/48021/fd (opendir: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48022 bstore_kv ceph cwd unknown /proc/46682/task/48022/cwd (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48022 bstore_kv ceph rtd unknown /proc/46682/task/48022/root (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48022 bstore_kv ceph txt unknown /proc/46682/task/48022/exe (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48022 bstore_kv ceph NOFD /proc/46682/task/48022/fd (opendir: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48023 bstore_me ceph cwd unknown /proc/46682/task/48023/cwd (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48023 bstore_me ceph rtd unknown /proc/46682/task/48023/root (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48023 bstore_me ceph txt unknown /proc/46682/task/48023/exe (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48023 bstore_me ceph NOFD /proc/46682/task/48023/fd (opendir: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48024 ms_dispat ceph cwd unknown /proc/46682/task/48024/cwd (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48024 ms_dispat ceph rtd unknown /proc/46682/task/48024/root (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48024 ms_dispat ceph txt unknown /proc/46682/task/48024/exe (readlink: Permission denied) 2024-03-16T08:19:37.771 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48024 ms_dispat ceph NOFD /proc/46682/task/48024/fd (opendir: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48025 ms_local ceph cwd unknown /proc/46682/task/48025/cwd (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48025 ms_local ceph rtd unknown /proc/46682/task/48025/root (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48025 ms_local ceph txt unknown /proc/46682/task/48025/exe (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48025 ms_local ceph NOFD /proc/46682/task/48025/fd (opendir: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48026 safe_time ceph cwd unknown /proc/46682/task/48026/cwd (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48026 safe_time ceph rtd unknown /proc/46682/task/48026/root (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48026 safe_time ceph txt unknown /proc/46682/task/48026/exe (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48026 safe_time ceph NOFD /proc/46682/task/48026/fd (opendir: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48027 safe_time ceph cwd unknown /proc/46682/task/48027/cwd (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48027 safe_time ceph rtd unknown /proc/46682/task/48027/root (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48027 safe_time ceph txt unknown /proc/46682/task/48027/exe (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48027 safe_time ceph NOFD /proc/46682/task/48027/fd (opendir: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48028 ms_dispat ceph cwd unknown /proc/46682/task/48028/cwd (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48028 ms_dispat ceph rtd unknown /proc/46682/task/48028/root (readlink: Permission denied) 2024-03-16T08:19:37.772 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48028 ms_dispat ceph txt unknown /proc/46682/task/48028/exe (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48028 ms_dispat ceph NOFD /proc/46682/task/48028/fd (opendir: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48029 ms_local ceph cwd unknown /proc/46682/task/48029/cwd (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48029 ms_local ceph rtd unknown /proc/46682/task/48029/root (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48029 ms_local ceph txt unknown /proc/46682/task/48029/exe (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48029 ms_local ceph NOFD /proc/46682/task/48029/fd (opendir: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48030 ms_dispat ceph cwd unknown /proc/46682/task/48030/cwd (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48030 ms_dispat ceph rtd unknown /proc/46682/task/48030/root (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48030 ms_dispat ceph txt unknown /proc/46682/task/48030/exe (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48030 ms_dispat ceph NOFD /proc/46682/task/48030/fd (opendir: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48031 ms_local ceph cwd unknown /proc/46682/task/48031/cwd (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48031 ms_local ceph rtd unknown /proc/46682/task/48031/root (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48031 ms_local ceph txt unknown /proc/46682/task/48031/exe (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48031 ms_local ceph NOFD /proc/46682/task/48031/fd (opendir: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48032 ms_dispat ceph cwd unknown /proc/46682/task/48032/cwd (readlink: Permission denied) 2024-03-16T08:19:37.773 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48032 ms_dispat ceph rtd unknown /proc/46682/task/48032/root (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48032 ms_dispat ceph txt unknown /proc/46682/task/48032/exe (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48032 ms_dispat ceph NOFD /proc/46682/task/48032/fd (opendir: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48033 ms_local ceph cwd unknown /proc/46682/task/48033/cwd (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48033 ms_local ceph rtd unknown /proc/46682/task/48033/root (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48033 ms_local ceph txt unknown /proc/46682/task/48033/exe (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48033 ms_local ceph NOFD /proc/46682/task/48033/fd (opendir: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48034 ms_dispat ceph cwd unknown /proc/46682/task/48034/cwd (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48034 ms_dispat ceph rtd unknown /proc/46682/task/48034/root (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48034 ms_dispat ceph txt unknown /proc/46682/task/48034/exe (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48034 ms_dispat ceph NOFD /proc/46682/task/48034/fd (opendir: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48035 ms_local ceph cwd unknown /proc/46682/task/48035/cwd (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48035 ms_local ceph rtd unknown /proc/46682/task/48035/root (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48035 ms_local ceph txt unknown /proc/46682/task/48035/exe (readlink: Permission denied) 2024-03-16T08:19:37.774 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48035 ms_local ceph NOFD /proc/46682/task/48035/fd (opendir: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48036 ms_dispat ceph cwd unknown /proc/46682/task/48036/cwd (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48036 ms_dispat ceph rtd unknown /proc/46682/task/48036/root (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48036 ms_dispat ceph txt unknown /proc/46682/task/48036/exe (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48036 ms_dispat ceph NOFD /proc/46682/task/48036/fd (opendir: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48037 ms_local ceph cwd unknown /proc/46682/task/48037/cwd (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48037 ms_local ceph rtd unknown /proc/46682/task/48037/root (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48037 ms_local ceph txt unknown /proc/46682/task/48037/exe (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48037 ms_local ceph NOFD /proc/46682/task/48037/fd (opendir: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48038 ms_dispat ceph cwd unknown /proc/46682/task/48038/cwd (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48038 ms_dispat ceph rtd unknown /proc/46682/task/48038/root (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48038 ms_dispat ceph txt unknown /proc/46682/task/48038/exe (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48038 ms_dispat ceph NOFD /proc/46682/task/48038/fd (opendir: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48039 ms_local ceph cwd unknown /proc/46682/task/48039/cwd (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48039 ms_local ceph rtd unknown /proc/46682/task/48039/root (readlink: Permission denied) 2024-03-16T08:19:37.775 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48039 ms_local ceph txt unknown /proc/46682/task/48039/exe (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48039 ms_local ceph NOFD /proc/46682/task/48039/fd (opendir: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48040 fn_anonym ceph cwd unknown /proc/46682/task/48040/cwd (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48040 fn_anonym ceph rtd unknown /proc/46682/task/48040/root (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48040 fn_anonym ceph txt unknown /proc/46682/task/48040/exe (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48040 fn_anonym ceph NOFD /proc/46682/task/48040/fd (opendir: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48041 finisher ceph cwd unknown /proc/46682/task/48041/cwd (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48041 finisher ceph rtd unknown /proc/46682/task/48041/root (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48041 finisher ceph txt unknown /proc/46682/task/48041/exe (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48041 finisher ceph NOFD /proc/46682/task/48041/fd (opendir: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48042 safe_time ceph cwd unknown /proc/46682/task/48042/cwd (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48042 safe_time ceph rtd unknown /proc/46682/task/48042/root (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48042 safe_time ceph txt unknown /proc/46682/task/48042/exe (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48042 safe_time ceph NOFD /proc/46682/task/48042/fd (opendir: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48043 safe_time ceph cwd unknown /proc/46682/task/48043/cwd (readlink: Permission denied) 2024-03-16T08:19:37.776 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48043 safe_time ceph rtd unknown /proc/46682/task/48043/root (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48043 safe_time ceph txt unknown /proc/46682/task/48043/exe (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48043 safe_time ceph NOFD /proc/46682/task/48043/fd (opendir: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48044 ceph-osd ceph cwd unknown /proc/46682/task/48044/cwd (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48044 ceph-osd ceph rtd unknown /proc/46682/task/48044/root (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48044 ceph-osd ceph txt unknown /proc/46682/task/48044/exe (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48044 ceph-osd ceph NOFD /proc/46682/task/48044/fd (opendir: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48045 osd_srv_a ceph cwd unknown /proc/46682/task/48045/cwd (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48045 osd_srv_a ceph rtd unknown /proc/46682/task/48045/root (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48045 osd_srv_a ceph txt unknown /proc/46682/task/48045/exe (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48045 osd_srv_a ceph NOFD /proc/46682/task/48045/fd (opendir: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48046 tp_osd_tp ceph cwd unknown /proc/46682/task/48046/cwd (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48046 tp_osd_tp ceph rtd unknown /proc/46682/task/48046/root (readlink: Permission denied) 2024-03-16T08:19:37.777 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48046 tp_osd_tp ceph txt unknown /proc/46682/task/48046/exe (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48046 tp_osd_tp ceph NOFD /proc/46682/task/48046/fd (opendir: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48047 tp_osd_tp ceph cwd unknown /proc/46682/task/48047/cwd (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48047 tp_osd_tp ceph rtd unknown /proc/46682/task/48047/root (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48047 tp_osd_tp ceph txt unknown /proc/46682/task/48047/exe (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48047 tp_osd_tp ceph NOFD /proc/46682/task/48047/fd (opendir: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48048 tp_osd_tp ceph cwd unknown /proc/46682/task/48048/cwd (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48048 tp_osd_tp ceph rtd unknown /proc/46682/task/48048/root (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48048 tp_osd_tp ceph txt unknown /proc/46682/task/48048/exe (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48048 tp_osd_tp ceph NOFD /proc/46682/task/48048/fd (opendir: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48049 tp_osd_tp ceph cwd unknown /proc/46682/task/48049/cwd (readlink: Permission denied) 2024-03-16T08:19:37.778 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48049 tp_osd_tp ceph rtd unknown /proc/46682/task/48049/root (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48049 tp_osd_tp ceph txt unknown /proc/46682/task/48049/exe (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48049 tp_osd_tp ceph NOFD /proc/46682/task/48049/fd (opendir: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48050 tp_osd_tp ceph cwd unknown /proc/46682/task/48050/cwd (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48050 tp_osd_tp ceph rtd unknown /proc/46682/task/48050/root (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48050 tp_osd_tp ceph txt unknown /proc/46682/task/48050/exe (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48050 tp_osd_tp ceph NOFD /proc/46682/task/48050/fd (opendir: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48051 tp_osd_tp ceph cwd unknown /proc/46682/task/48051/cwd (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48051 tp_osd_tp ceph rtd unknown /proc/46682/task/48051/root (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48051 tp_osd_tp ceph txt unknown /proc/46682/task/48051/exe (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48051 tp_osd_tp ceph NOFD /proc/46682/task/48051/fd (opendir: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48052 tp_osd_tp ceph cwd unknown /proc/46682/task/48052/cwd (readlink: Permission denied) 2024-03-16T08:19:37.779 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48052 tp_osd_tp ceph rtd unknown /proc/46682/task/48052/root (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48052 tp_osd_tp ceph txt unknown /proc/46682/task/48052/exe (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48052 tp_osd_tp ceph NOFD /proc/46682/task/48052/fd (opendir: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48053 tp_osd_tp ceph cwd unknown /proc/46682/task/48053/cwd (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48053 tp_osd_tp ceph rtd unknown /proc/46682/task/48053/root (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48053 tp_osd_tp ceph txt unknown /proc/46682/task/48053/exe (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48053 tp_osd_tp ceph NOFD /proc/46682/task/48053/fd (opendir: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48054 tp_osd_tp ceph cwd unknown /proc/46682/task/48054/cwd (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48054 tp_osd_tp ceph rtd unknown /proc/46682/task/48054/root (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48054 tp_osd_tp ceph txt unknown /proc/46682/task/48054/exe (readlink: Permission denied) 2024-03-16T08:19:37.780 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48054 tp_osd_tp ceph NOFD /proc/46682/task/48054/fd (opendir: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48055 tp_osd_tp ceph cwd unknown /proc/46682/task/48055/cwd (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48055 tp_osd_tp ceph rtd unknown /proc/46682/task/48055/root (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48055 tp_osd_tp ceph txt unknown /proc/46682/task/48055/exe (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48055 tp_osd_tp ceph NOFD /proc/46682/task/48055/fd (opendir: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48056 tp_osd_tp ceph cwd unknown /proc/46682/task/48056/cwd (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48056 tp_osd_tp ceph rtd unknown /proc/46682/task/48056/root (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48056 tp_osd_tp ceph txt unknown /proc/46682/task/48056/exe (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48056 tp_osd_tp ceph NOFD /proc/46682/task/48056/fd (opendir: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48057 tp_osd_tp ceph cwd unknown /proc/46682/task/48057/cwd (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48057 tp_osd_tp ceph rtd unknown /proc/46682/task/48057/root (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48057 tp_osd_tp ceph txt unknown /proc/46682/task/48057/exe (readlink: Permission denied) 2024-03-16T08:19:37.781 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48057 tp_osd_tp ceph NOFD /proc/46682/task/48057/fd (opendir: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48058 tp_osd_tp ceph cwd unknown /proc/46682/task/48058/cwd (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48058 tp_osd_tp ceph rtd unknown /proc/46682/task/48058/root (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48058 tp_osd_tp ceph txt unknown /proc/46682/task/48058/exe (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48058 tp_osd_tp ceph NOFD /proc/46682/task/48058/fd (opendir: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48059 tp_osd_tp ceph cwd unknown /proc/46682/task/48059/cwd (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48059 tp_osd_tp ceph rtd unknown /proc/46682/task/48059/root (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48059 tp_osd_tp ceph txt unknown /proc/46682/task/48059/exe (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48059 tp_osd_tp ceph NOFD /proc/46682/task/48059/fd (opendir: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48060 tp_osd_tp ceph cwd unknown /proc/46682/task/48060/cwd (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48060 tp_osd_tp ceph rtd unknown /proc/46682/task/48060/root (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48060 tp_osd_tp ceph txt unknown /proc/46682/task/48060/exe (readlink: Permission denied) 2024-03-16T08:19:37.782 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48060 tp_osd_tp ceph NOFD /proc/46682/task/48060/fd (opendir: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48061 tp_osd_tp ceph cwd unknown /proc/46682/task/48061/cwd (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48061 tp_osd_tp ceph rtd unknown /proc/46682/task/48061/root (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48061 tp_osd_tp ceph txt unknown /proc/46682/task/48061/exe (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48061 tp_osd_tp ceph NOFD /proc/46682/task/48061/fd (opendir: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48062 osd_srv_h ceph cwd unknown /proc/46682/task/48062/cwd (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48062 osd_srv_h ceph rtd unknown /proc/46682/task/48062/root (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48062 osd_srv_h ceph txt unknown /proc/46682/task/48062/exe (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 46682 48062 osd_srv_h ceph NOFD /proc/46682/task/48062/fd (opendir: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 48077 root cwd unknown /proc/48077/cwd (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 48077 root rtd unknown /proc/48077/root (readlink: Permission denied) 2024-03-16T08:19:37.783 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 48077 root txt unknown /proc/48077/exe (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 48077 root NOFD /proc/48077/fd (opendir: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 48108 root cwd unknown /proc/48108/cwd (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 48108 root rtd unknown /proc/48108/root (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 48108 root txt unknown /proc/48108/exe (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 48108 root NOFD /proc/48108/fd (opendir: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 50465 root cwd unknown /proc/50465/cwd (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 50465 root rtd unknown /proc/50465/root (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 50465 root txt unknown /proc/50465/exe (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 50465 root NOFD /proc/50465/fd (opendir: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 53108 root cwd unknown /proc/53108/cwd (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 53108 root rtd unknown /proc/53108/root (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 53108 root txt unknown /proc/53108/exe (readlink: Permission denied) 2024-03-16T08:19:37.784 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/6 53108 root NOFD /proc/53108/fd (opendir: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 53378 root cwd unknown /proc/53378/cwd (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 53378 root rtd unknown /proc/53378/root (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 53378 root txt unknown /proc/53378/exe (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 53378 root NOFD /proc/53378/fd (opendir: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 53380 root cwd unknown /proc/53380/cwd (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 53380 root rtd unknown /proc/53380/root (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 53380 root txt unknown /proc/53380/exe (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 53380 root NOFD /proc/53380/fd (opendir: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 ceph cwd unknown /proc/53382/cwd (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 ceph rtd unknown /proc/53382/root (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 ceph txt unknown /proc/53382/exe (readlink: Permission denied) 2024-03-16T08:19:37.785 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 ceph NOFD /proc/53382/fd (opendir: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53383 log ceph cwd unknown /proc/53382/task/53383/cwd (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53383 log ceph rtd unknown /proc/53382/task/53383/root (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53383 log ceph txt unknown /proc/53382/task/53383/exe (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53383 log ceph NOFD /proc/53382/task/53383/fd (opendir: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53385 msgr-work ceph cwd unknown /proc/53382/task/53385/cwd (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53385 msgr-work ceph rtd unknown /proc/53382/task/53385/root (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53385 msgr-work ceph txt unknown /proc/53382/task/53385/exe (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53385 msgr-work ceph NOFD /proc/53382/task/53385/fd (opendir: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53386 msgr-work ceph cwd unknown /proc/53382/task/53386/cwd (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53386 msgr-work ceph rtd unknown /proc/53382/task/53386/root (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53386 msgr-work ceph txt unknown /proc/53382/task/53386/exe (readlink: Permission denied) 2024-03-16T08:19:37.786 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53386 msgr-work ceph NOFD /proc/53382/task/53386/fd (opendir: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53387 msgr-work ceph cwd unknown /proc/53382/task/53387/cwd (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53387 msgr-work ceph rtd unknown /proc/53382/task/53387/root (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53387 msgr-work ceph txt unknown /proc/53382/task/53387/exe (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53387 msgr-work ceph NOFD /proc/53382/task/53387/fd (opendir: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53391 service ceph cwd unknown /proc/53382/task/53391/cwd (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53391 service ceph rtd unknown /proc/53382/task/53391/root (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53391 service ceph txt unknown /proc/53382/task/53391/exe (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53391 service ceph NOFD /proc/53382/task/53391/fd (opendir: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53392 admin_soc ceph cwd unknown /proc/53382/task/53392/cwd (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53392 admin_soc ceph rtd unknown /proc/53382/task/53392/root (readlink: Permission denied) 2024-03-16T08:19:37.787 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53392 admin_soc ceph txt unknown /proc/53382/task/53392/exe (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53392 admin_soc ceph NOFD /proc/53382/task/53392/fd (opendir: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53534 signal_ha ceph cwd unknown /proc/53382/task/53534/cwd (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53534 signal_ha ceph rtd unknown /proc/53382/task/53534/root (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53534 signal_ha ceph txt unknown /proc/53382/task/53534/exe (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53534 signal_ha ceph NOFD /proc/53382/task/53534/fd (opendir: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53535 io_contex ceph cwd unknown /proc/53382/task/53535/cwd (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53535 io_contex ceph rtd unknown /proc/53382/task/53535/root (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53535 io_contex ceph txt unknown /proc/53382/task/53535/exe (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53535 io_contex ceph NOFD /proc/53382/task/53535/fd (opendir: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53536 io_contex ceph cwd unknown /proc/53382/task/53536/cwd (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53536 io_contex ceph rtd unknown /proc/53382/task/53536/root (readlink: Permission denied) 2024-03-16T08:19:37.788 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53536 io_contex ceph txt unknown /proc/53382/task/53536/exe (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53536 io_contex ceph NOFD /proc/53382/task/53536/fd (opendir: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53653 OpHistory ceph cwd unknown /proc/53382/task/53653/cwd (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53653 OpHistory ceph rtd unknown /proc/53382/task/53653/root (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53653 OpHistory ceph txt unknown /proc/53382/task/53653/exe (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53653 OpHistory ceph NOFD /proc/53382/task/53653/fd (opendir: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53654 ceph_time ceph cwd unknown /proc/53382/task/53654/cwd (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53654 ceph_time ceph rtd unknown /proc/53382/task/53654/root (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53654 ceph_time ceph txt unknown /proc/53382/task/53654/exe (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53654 ceph_time ceph NOFD /proc/53382/task/53654/fd (opendir: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53859 ceph-osd ceph cwd unknown /proc/53382/task/53859/cwd (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53859 ceph-osd ceph rtd unknown /proc/53382/task/53859/root (readlink: Permission denied) 2024-03-16T08:19:37.789 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53859 ceph-osd ceph txt unknown /proc/53382/task/53859/exe (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53859 ceph-osd ceph NOFD /proc/53382/task/53859/fd (opendir: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53860 safe_time ceph cwd unknown /proc/53382/task/53860/cwd (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53860 safe_time ceph rtd unknown /proc/53382/task/53860/root (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53860 safe_time ceph txt unknown /proc/53382/task/53860/exe (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53860 safe_time ceph NOFD /proc/53382/task/53860/fd (opendir: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53861 safe_time ceph cwd unknown /proc/53382/task/53861/cwd (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53861 safe_time ceph rtd unknown /proc/53382/task/53861/root (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53861 safe_time ceph txt unknown /proc/53382/task/53861/exe (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53861 safe_time ceph NOFD /proc/53382/task/53861/fd (opendir: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53862 safe_time ceph cwd unknown /proc/53382/task/53862/cwd (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53862 safe_time ceph rtd unknown /proc/53382/task/53862/root (readlink: Permission denied) 2024-03-16T08:19:37.790 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53862 safe_time ceph txt unknown /proc/53382/task/53862/exe (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53862 safe_time ceph NOFD /proc/53382/task/53862/fd (opendir: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53864 safe_time ceph cwd unknown /proc/53382/task/53864/cwd (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53864 safe_time ceph rtd unknown /proc/53382/task/53864/root (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53864 safe_time ceph txt unknown /proc/53382/task/53864/exe (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53864 safe_time ceph NOFD /proc/53382/task/53864/fd (opendir: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53865 fn_anonym ceph cwd unknown /proc/53382/task/53865/cwd (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53865 fn_anonym ceph rtd unknown /proc/53382/task/53865/root (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53865 fn_anonym ceph txt unknown /proc/53382/task/53865/exe (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53865 fn_anonym ceph NOFD /proc/53382/task/53865/fd (opendir: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53868 rocksdb:l ceph cwd unknown /proc/53382/task/53868/cwd (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53868 rocksdb:l ceph rtd unknown /proc/53382/task/53868/root (readlink: Permission denied) 2024-03-16T08:19:37.791 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53868 rocksdb:l ceph txt unknown /proc/53382/task/53868/exe (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53868 rocksdb:l ceph NOFD /proc/53382/task/53868/fd (opendir: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53869 rocksdb:l ceph cwd unknown /proc/53382/task/53869/cwd (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53869 rocksdb:l ceph rtd unknown /proc/53382/task/53869/root (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53869 rocksdb:l ceph txt unknown /proc/53382/task/53869/exe (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53869 rocksdb:l ceph NOFD /proc/53382/task/53869/fd (opendir: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53870 rocksdb:l ceph cwd unknown /proc/53382/task/53870/cwd (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53870 rocksdb:l ceph rtd unknown /proc/53382/task/53870/root (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53870 rocksdb:l ceph txt unknown /proc/53382/task/53870/exe (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53870 rocksdb:l ceph NOFD /proc/53382/task/53870/fd (opendir: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53871 rocksdb:h ceph cwd unknown /proc/53382/task/53871/cwd (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53871 rocksdb:h ceph rtd unknown /proc/53382/task/53871/root (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53871 rocksdb:h ceph txt unknown /proc/53382/task/53871/exe (readlink: Permission denied) 2024-03-16T08:19:37.792 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 53871 rocksdb:h ceph NOFD /proc/53382/task/53871/fd (opendir: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54277 bstore_ai ceph cwd unknown /proc/53382/task/54277/cwd (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54277 bstore_ai ceph rtd unknown /proc/53382/task/54277/root (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54277 bstore_ai ceph txt unknown /proc/53382/task/54277/exe (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54277 bstore_ai ceph NOFD /proc/53382/task/54277/fd (opendir: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54540 bstore_ai ceph cwd unknown /proc/53382/task/54540/cwd (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54540 bstore_ai ceph rtd unknown /proc/53382/task/54540/root (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54540 bstore_ai ceph txt unknown /proc/53382/task/54540/exe (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54540 bstore_ai ceph NOFD /proc/53382/task/54540/fd (opendir: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54722 ceph-osd ceph cwd unknown /proc/53382/task/54722/cwd (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54722 ceph-osd ceph rtd unknown /proc/53382/task/54722/root (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54722 ceph-osd ceph txt unknown /proc/53382/task/54722/exe (readlink: Permission denied) 2024-03-16T08:19:37.793 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54722 ceph-osd ceph NOFD /proc/53382/task/54722/fd (opendir: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54723 cfin ceph cwd unknown /proc/53382/task/54723/cwd (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54723 cfin ceph rtd unknown /proc/53382/task/54723/root (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54723 cfin ceph txt unknown /proc/53382/task/54723/exe (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54723 cfin ceph NOFD /proc/53382/task/54723/fd (opendir: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54724 bstore_kv ceph cwd unknown /proc/53382/task/54724/cwd (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54724 bstore_kv ceph rtd unknown /proc/53382/task/54724/root (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54724 bstore_kv ceph txt unknown /proc/53382/task/54724/exe (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54724 bstore_kv ceph NOFD /proc/53382/task/54724/fd (opendir: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54725 bstore_kv ceph cwd unknown /proc/53382/task/54725/cwd (readlink: Permission denied) 2024-03-16T08:19:37.794 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54725 bstore_kv ceph rtd unknown /proc/53382/task/54725/root (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54725 bstore_kv ceph txt unknown /proc/53382/task/54725/exe (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54725 bstore_kv ceph NOFD /proc/53382/task/54725/fd (opendir: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54726 bstore_me ceph cwd unknown /proc/53382/task/54726/cwd (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54726 bstore_me ceph rtd unknown /proc/53382/task/54726/root (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54726 bstore_me ceph txt unknown /proc/53382/task/54726/exe (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54726 bstore_me ceph NOFD /proc/53382/task/54726/fd (opendir: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54727 ms_dispat ceph cwd unknown /proc/53382/task/54727/cwd (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54727 ms_dispat ceph rtd unknown /proc/53382/task/54727/root (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54727 ms_dispat ceph txt unknown /proc/53382/task/54727/exe (readlink: Permission denied) 2024-03-16T08:19:37.795 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54727 ms_dispat ceph NOFD /proc/53382/task/54727/fd (opendir: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54728 ms_local ceph cwd unknown /proc/53382/task/54728/cwd (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54728 ms_local ceph rtd unknown /proc/53382/task/54728/root (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54728 ms_local ceph txt unknown /proc/53382/task/54728/exe (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54728 ms_local ceph NOFD /proc/53382/task/54728/fd (opendir: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54729 safe_time ceph cwd unknown /proc/53382/task/54729/cwd (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54729 safe_time ceph rtd unknown /proc/53382/task/54729/root (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54729 safe_time ceph txt unknown /proc/53382/task/54729/exe (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54729 safe_time ceph NOFD /proc/53382/task/54729/fd (opendir: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54730 safe_time ceph cwd unknown /proc/53382/task/54730/cwd (readlink: Permission denied) 2024-03-16T08:19:37.796 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54730 safe_time ceph rtd unknown /proc/53382/task/54730/root (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54730 safe_time ceph txt unknown /proc/53382/task/54730/exe (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54730 safe_time ceph NOFD /proc/53382/task/54730/fd (opendir: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54731 ms_dispat ceph cwd unknown /proc/53382/task/54731/cwd (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54731 ms_dispat ceph rtd unknown /proc/53382/task/54731/root (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54731 ms_dispat ceph txt unknown /proc/53382/task/54731/exe (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54731 ms_dispat ceph NOFD /proc/53382/task/54731/fd (opendir: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54732 ms_local ceph cwd unknown /proc/53382/task/54732/cwd (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54732 ms_local ceph rtd unknown /proc/53382/task/54732/root (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54732 ms_local ceph txt unknown /proc/53382/task/54732/exe (readlink: Permission denied) 2024-03-16T08:19:37.797 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54732 ms_local ceph NOFD /proc/53382/task/54732/fd (opendir: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54733 ms_dispat ceph cwd unknown /proc/53382/task/54733/cwd (readlink: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54733 ms_dispat ceph rtd unknown /proc/53382/task/54733/root (readlink: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54733 ms_dispat ceph txt unknown /proc/53382/task/54733/exe (readlink: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54733 ms_dispat ceph NOFD /proc/53382/task/54733/fd (opendir: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54734 ms_local ceph cwd unknown /proc/53382/task/54734/cwd (readlink: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54734 ms_local ceph rtd unknown /proc/53382/task/54734/root (readlink: Permission denied) 2024-03-16T08:19:37.798 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54734 ms_local ceph txt unknown /proc/53382/task/54734/exe (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54734 ms_local ceph NOFD /proc/53382/task/54734/fd (opendir: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54735 ms_dispat ceph cwd unknown /proc/53382/task/54735/cwd (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54735 ms_dispat ceph rtd unknown /proc/53382/task/54735/root (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54735 ms_dispat ceph txt unknown /proc/53382/task/54735/exe (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54735 ms_dispat ceph NOFD /proc/53382/task/54735/fd (opendir: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54736 ms_local ceph cwd unknown /proc/53382/task/54736/cwd (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54736 ms_local ceph rtd unknown /proc/53382/task/54736/root (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54736 ms_local ceph txt unknown /proc/53382/task/54736/exe (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54736 ms_local ceph NOFD /proc/53382/task/54736/fd (opendir: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54737 ms_dispat ceph cwd unknown /proc/53382/task/54737/cwd (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54737 ms_dispat ceph rtd unknown /proc/53382/task/54737/root (readlink: Permission denied) 2024-03-16T08:19:37.799 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54737 ms_dispat ceph txt unknown /proc/53382/task/54737/exe (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54737 ms_dispat ceph NOFD /proc/53382/task/54737/fd (opendir: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54738 ms_local ceph cwd unknown /proc/53382/task/54738/cwd (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54738 ms_local ceph rtd unknown /proc/53382/task/54738/root (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54738 ms_local ceph txt unknown /proc/53382/task/54738/exe (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54738 ms_local ceph NOFD /proc/53382/task/54738/fd (opendir: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54739 ms_dispat ceph cwd unknown /proc/53382/task/54739/cwd (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54739 ms_dispat ceph rtd unknown /proc/53382/task/54739/root (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54739 ms_dispat ceph txt unknown /proc/53382/task/54739/exe (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54739 ms_dispat ceph NOFD /proc/53382/task/54739/fd (opendir: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54740 ms_local ceph cwd unknown /proc/53382/task/54740/cwd (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54740 ms_local ceph rtd unknown /proc/53382/task/54740/root (readlink: Permission denied) 2024-03-16T08:19:37.800 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54740 ms_local ceph txt unknown /proc/53382/task/54740/exe (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54740 ms_local ceph NOFD /proc/53382/task/54740/fd (opendir: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54741 ms_dispat ceph cwd unknown /proc/53382/task/54741/cwd (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54741 ms_dispat ceph rtd unknown /proc/53382/task/54741/root (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54741 ms_dispat ceph txt unknown /proc/53382/task/54741/exe (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54741 ms_dispat ceph NOFD /proc/53382/task/54741/fd (opendir: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54742 ms_local ceph cwd unknown /proc/53382/task/54742/cwd (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54742 ms_local ceph rtd unknown /proc/53382/task/54742/root (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54742 ms_local ceph txt unknown /proc/53382/task/54742/exe (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54742 ms_local ceph NOFD /proc/53382/task/54742/fd (opendir: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54743 fn_anonym ceph cwd unknown /proc/53382/task/54743/cwd (readlink: Permission denied) 2024-03-16T08:19:37.801 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54743 fn_anonym ceph rtd unknown /proc/53382/task/54743/root (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54743 fn_anonym ceph txt unknown /proc/53382/task/54743/exe (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54743 fn_anonym ceph NOFD /proc/53382/task/54743/fd (opendir: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54744 finisher ceph cwd unknown /proc/53382/task/54744/cwd (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54744 finisher ceph rtd unknown /proc/53382/task/54744/root (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54744 finisher ceph txt unknown /proc/53382/task/54744/exe (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54744 finisher ceph NOFD /proc/53382/task/54744/fd (opendir: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54745 safe_time ceph cwd unknown /proc/53382/task/54745/cwd (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54745 safe_time ceph rtd unknown /proc/53382/task/54745/root (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54745 safe_time ceph txt unknown /proc/53382/task/54745/exe (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54745 safe_time ceph NOFD /proc/53382/task/54745/fd (opendir: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54746 safe_time ceph cwd unknown /proc/53382/task/54746/cwd (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54746 safe_time ceph rtd unknown /proc/53382/task/54746/root (readlink: Permission denied) 2024-03-16T08:19:37.802 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54746 safe_time ceph txt unknown /proc/53382/task/54746/exe (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54746 safe_time ceph NOFD /proc/53382/task/54746/fd (opendir: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54747 ceph-osd ceph cwd unknown /proc/53382/task/54747/cwd (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54747 ceph-osd ceph rtd unknown /proc/53382/task/54747/root (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54747 ceph-osd ceph txt unknown /proc/53382/task/54747/exe (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54747 ceph-osd ceph NOFD /proc/53382/task/54747/fd (opendir: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54748 osd_srv_a ceph cwd unknown /proc/53382/task/54748/cwd (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54748 osd_srv_a ceph rtd unknown /proc/53382/task/54748/root (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54748 osd_srv_a ceph txt unknown /proc/53382/task/54748/exe (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54748 osd_srv_a ceph NOFD /proc/53382/task/54748/fd (opendir: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54749 tp_osd_tp ceph cwd unknown /proc/53382/task/54749/cwd (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54749 tp_osd_tp ceph rtd unknown /proc/53382/task/54749/root (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54749 tp_osd_tp ceph txt unknown /proc/53382/task/54749/exe (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54749 tp_osd_tp ceph NOFD /proc/53382/task/54749/fd (opendir: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54750 tp_osd_tp ceph cwd unknown /proc/53382/task/54750/cwd (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54750 tp_osd_tp ceph rtd unknown /proc/53382/task/54750/root (readlink: Permission denied) 2024-03-16T08:19:37.803 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54750 tp_osd_tp ceph txt unknown /proc/53382/task/54750/exe (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54750 tp_osd_tp ceph NOFD /proc/53382/task/54750/fd (opendir: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54751 tp_osd_tp ceph cwd unknown /proc/53382/task/54751/cwd (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54751 tp_osd_tp ceph rtd unknown /proc/53382/task/54751/root (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54751 tp_osd_tp ceph txt unknown /proc/53382/task/54751/exe (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54751 tp_osd_tp ceph NOFD /proc/53382/task/54751/fd (opendir: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54752 tp_osd_tp ceph cwd unknown /proc/53382/task/54752/cwd (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54752 tp_osd_tp ceph rtd unknown /proc/53382/task/54752/root (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54752 tp_osd_tp ceph txt unknown /proc/53382/task/54752/exe (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54752 tp_osd_tp ceph NOFD /proc/53382/task/54752/fd (opendir: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54753 tp_osd_tp ceph cwd unknown /proc/53382/task/54753/cwd (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54753 tp_osd_tp ceph rtd unknown /proc/53382/task/54753/root (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54753 tp_osd_tp ceph txt unknown /proc/53382/task/54753/exe (readlink: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54753 tp_osd_tp ceph NOFD /proc/53382/task/54753/fd (opendir: Permission denied) 2024-03-16T08:19:37.804 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54754 tp_osd_tp ceph cwd unknown /proc/53382/task/54754/cwd (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54754 tp_osd_tp ceph rtd unknown /proc/53382/task/54754/root (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54754 tp_osd_tp ceph txt unknown /proc/53382/task/54754/exe (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54754 tp_osd_tp ceph NOFD /proc/53382/task/54754/fd (opendir: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54755 tp_osd_tp ceph cwd unknown /proc/53382/task/54755/cwd (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54755 tp_osd_tp ceph rtd unknown /proc/53382/task/54755/root (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54755 tp_osd_tp ceph txt unknown /proc/53382/task/54755/exe (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54755 tp_osd_tp ceph NOFD /proc/53382/task/54755/fd (opendir: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54756 tp_osd_tp ceph cwd unknown /proc/53382/task/54756/cwd (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54756 tp_osd_tp ceph rtd unknown /proc/53382/task/54756/root (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54756 tp_osd_tp ceph txt unknown /proc/53382/task/54756/exe (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54756 tp_osd_tp ceph NOFD /proc/53382/task/54756/fd (opendir: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54757 tp_osd_tp ceph cwd unknown /proc/53382/task/54757/cwd (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54757 tp_osd_tp ceph rtd unknown /proc/53382/task/54757/root (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54757 tp_osd_tp ceph txt unknown /proc/53382/task/54757/exe (readlink: Permission denied) 2024-03-16T08:19:37.805 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54757 tp_osd_tp ceph NOFD /proc/53382/task/54757/fd (opendir: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54758 tp_osd_tp ceph cwd unknown /proc/53382/task/54758/cwd (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54758 tp_osd_tp ceph rtd unknown /proc/53382/task/54758/root (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54758 tp_osd_tp ceph txt unknown /proc/53382/task/54758/exe (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54758 tp_osd_tp ceph NOFD /proc/53382/task/54758/fd (opendir: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54759 tp_osd_tp ceph cwd unknown /proc/53382/task/54759/cwd (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54759 tp_osd_tp ceph rtd unknown /proc/53382/task/54759/root (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54759 tp_osd_tp ceph txt unknown /proc/53382/task/54759/exe (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54759 tp_osd_tp ceph NOFD /proc/53382/task/54759/fd (opendir: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54760 tp_osd_tp ceph cwd unknown /proc/53382/task/54760/cwd (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54760 tp_osd_tp ceph rtd unknown /proc/53382/task/54760/root (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54760 tp_osd_tp ceph txt unknown /proc/53382/task/54760/exe (readlink: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54760 tp_osd_tp ceph NOFD /proc/53382/task/54760/fd (opendir: Permission denied) 2024-03-16T08:19:37.806 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54761 tp_osd_tp ceph cwd unknown /proc/53382/task/54761/cwd (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54761 tp_osd_tp ceph rtd unknown /proc/53382/task/54761/root (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54761 tp_osd_tp ceph txt unknown /proc/53382/task/54761/exe (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54761 tp_osd_tp ceph NOFD /proc/53382/task/54761/fd (opendir: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54762 tp_osd_tp ceph cwd unknown /proc/53382/task/54762/cwd (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54762 tp_osd_tp ceph rtd unknown /proc/53382/task/54762/root (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54762 tp_osd_tp ceph txt unknown /proc/53382/task/54762/exe (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54762 tp_osd_tp ceph NOFD /proc/53382/task/54762/fd (opendir: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54763 tp_osd_tp ceph cwd unknown /proc/53382/task/54763/cwd (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54763 tp_osd_tp ceph rtd unknown /proc/53382/task/54763/root (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54763 tp_osd_tp ceph txt unknown /proc/53382/task/54763/exe (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54763 tp_osd_tp ceph NOFD /proc/53382/task/54763/fd (opendir: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54764 tp_osd_tp ceph cwd unknown /proc/53382/task/54764/cwd (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54764 tp_osd_tp ceph rtd unknown /proc/53382/task/54764/root (readlink: Permission denied) 2024-03-16T08:19:37.807 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54764 tp_osd_tp ceph txt unknown /proc/53382/task/54764/exe (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54764 tp_osd_tp ceph NOFD /proc/53382/task/54764/fd (opendir: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54765 osd_srv_h ceph cwd unknown /proc/53382/task/54765/cwd (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54765 osd_srv_h ceph rtd unknown /proc/53382/task/54765/root (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54765 osd_srv_h ceph txt unknown /proc/53382/task/54765/exe (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 53382 54765 osd_srv_h ceph NOFD /proc/53382/task/54765/fd (opendir: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 54778 root cwd unknown /proc/54778/cwd (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 54778 root rtd unknown /proc/54778/root (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 54778 root txt unknown /proc/54778/exe (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 54778 root NOFD /proc/54778/fd (opendir: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 54811 root cwd unknown /proc/54811/cwd (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 54811 root rtd unknown /proc/54811/root (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 54811 root txt unknown /proc/54811/exe (readlink: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 54811 root NOFD /proc/54811/fd (opendir: Permission denied) 2024-03-16T08:19:37.808 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 59166 root cwd unknown /proc/59166/cwd (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 59166 root rtd unknown /proc/59166/root (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 59166 root txt unknown /proc/59166/exe (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/u 59166 root NOFD /proc/59166/fd (opendir: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 59889 root cwd unknown /proc/59889/cwd (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 59889 root rtd unknown /proc/59889/root (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 59889 root txt unknown /proc/59889/exe (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 59889 root NOFD /proc/59889/fd (opendir: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 59891 root cwd unknown /proc/59891/cwd (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 59891 root rtd unknown /proc/59891/root (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 59891 root txt unknown /proc/59891/exe (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 59891 root NOFD /proc/59891/fd (opendir: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 ceph cwd unknown /proc/59893/cwd (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 ceph rtd unknown /proc/59893/root (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 ceph txt unknown /proc/59893/exe (readlink: Permission denied) 2024-03-16T08:19:37.809 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 ceph NOFD /proc/59893/fd (opendir: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59894 log ceph cwd unknown /proc/59893/task/59894/cwd (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59894 log ceph rtd unknown /proc/59893/task/59894/root (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59894 log ceph txt unknown /proc/59893/task/59894/exe (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59894 log ceph NOFD /proc/59893/task/59894/fd (opendir: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59896 msgr-work ceph cwd unknown /proc/59893/task/59896/cwd (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59896 msgr-work ceph rtd unknown /proc/59893/task/59896/root (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59896 msgr-work ceph txt unknown /proc/59893/task/59896/exe (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59896 msgr-work ceph NOFD /proc/59893/task/59896/fd (opendir: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59897 msgr-work ceph cwd unknown /proc/59893/task/59897/cwd (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59897 msgr-work ceph rtd unknown /proc/59893/task/59897/root (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59897 msgr-work ceph txt unknown /proc/59893/task/59897/exe (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59897 msgr-work ceph NOFD /proc/59893/task/59897/fd (opendir: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59898 msgr-work ceph cwd unknown /proc/59893/task/59898/cwd (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59898 msgr-work ceph rtd unknown /proc/59893/task/59898/root (readlink: Permission denied) 2024-03-16T08:19:37.810 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59898 msgr-work ceph txt unknown /proc/59893/task/59898/exe (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59898 msgr-work ceph NOFD /proc/59893/task/59898/fd (opendir: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59902 service ceph cwd unknown /proc/59893/task/59902/cwd (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59902 service ceph rtd unknown /proc/59893/task/59902/root (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59902 service ceph txt unknown /proc/59893/task/59902/exe (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59902 service ceph NOFD /proc/59893/task/59902/fd (opendir: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59903 admin_soc ceph cwd unknown /proc/59893/task/59903/cwd (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59903 admin_soc ceph rtd unknown /proc/59893/task/59903/root (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59903 admin_soc ceph txt unknown /proc/59893/task/59903/exe (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 59903 admin_soc ceph NOFD /proc/59893/task/59903/fd (opendir: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60045 signal_ha ceph cwd unknown /proc/59893/task/60045/cwd (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60045 signal_ha ceph rtd unknown /proc/59893/task/60045/root (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60045 signal_ha ceph txt unknown /proc/59893/task/60045/exe (readlink: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60045 signal_ha ceph NOFD /proc/59893/task/60045/fd (opendir: Permission denied) 2024-03-16T08:19:37.811 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60046 io_contex ceph cwd unknown /proc/59893/task/60046/cwd (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60046 io_contex ceph rtd unknown /proc/59893/task/60046/root (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60046 io_contex ceph txt unknown /proc/59893/task/60046/exe (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60046 io_contex ceph NOFD /proc/59893/task/60046/fd (opendir: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60047 io_contex ceph cwd unknown /proc/59893/task/60047/cwd (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60047 io_contex ceph rtd unknown /proc/59893/task/60047/root (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60047 io_contex ceph txt unknown /proc/59893/task/60047/exe (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60047 io_contex ceph NOFD /proc/59893/task/60047/fd (opendir: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60160 OpHistory ceph cwd unknown /proc/59893/task/60160/cwd (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60160 OpHistory ceph rtd unknown /proc/59893/task/60160/root (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60160 OpHistory ceph txt unknown /proc/59893/task/60160/exe (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60160 OpHistory ceph NOFD /proc/59893/task/60160/fd (opendir: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60161 ceph_time ceph cwd unknown /proc/59893/task/60161/cwd (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60161 ceph_time ceph rtd unknown /proc/59893/task/60161/root (readlink: Permission denied) 2024-03-16T08:19:37.812 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60161 ceph_time ceph txt unknown /proc/59893/task/60161/exe (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60161 ceph_time ceph NOFD /proc/59893/task/60161/fd (opendir: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60335 ceph-osd ceph cwd unknown /proc/59893/task/60335/cwd (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60335 ceph-osd ceph rtd unknown /proc/59893/task/60335/root (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60335 ceph-osd ceph txt unknown /proc/59893/task/60335/exe (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60335 ceph-osd ceph NOFD /proc/59893/task/60335/fd (opendir: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60337 safe_time ceph cwd unknown /proc/59893/task/60337/cwd (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60337 safe_time ceph rtd unknown /proc/59893/task/60337/root (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60337 safe_time ceph txt unknown /proc/59893/task/60337/exe (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60337 safe_time ceph NOFD /proc/59893/task/60337/fd (opendir: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60338 safe_time ceph cwd unknown /proc/59893/task/60338/cwd (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60338 safe_time ceph rtd unknown /proc/59893/task/60338/root (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60338 safe_time ceph txt unknown /proc/59893/task/60338/exe (readlink: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60338 safe_time ceph NOFD /proc/59893/task/60338/fd (opendir: Permission denied) 2024-03-16T08:19:37.813 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60339 safe_time ceph cwd unknown /proc/59893/task/60339/cwd (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60339 safe_time ceph rtd unknown /proc/59893/task/60339/root (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60339 safe_time ceph txt unknown /proc/59893/task/60339/exe (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60339 safe_time ceph NOFD /proc/59893/task/60339/fd (opendir: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60340 safe_time ceph cwd unknown /proc/59893/task/60340/cwd (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60340 safe_time ceph rtd unknown /proc/59893/task/60340/root (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60340 safe_time ceph txt unknown /proc/59893/task/60340/exe (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60340 safe_time ceph NOFD /proc/59893/task/60340/fd (opendir: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60341 fn_anonym ceph cwd unknown /proc/59893/task/60341/cwd (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60341 fn_anonym ceph rtd unknown /proc/59893/task/60341/root (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60341 fn_anonym ceph txt unknown /proc/59893/task/60341/exe (readlink: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60341 fn_anonym ceph NOFD /proc/59893/task/60341/fd (opendir: Permission denied) 2024-03-16T08:19:37.814 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60344 rocksdb:l ceph cwd unknown /proc/59893/task/60344/cwd (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60344 rocksdb:l ceph rtd unknown /proc/59893/task/60344/root (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60344 rocksdb:l ceph txt unknown /proc/59893/task/60344/exe (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60344 rocksdb:l ceph NOFD /proc/59893/task/60344/fd (opendir: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60345 rocksdb:l ceph cwd unknown /proc/59893/task/60345/cwd (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60345 rocksdb:l ceph rtd unknown /proc/59893/task/60345/root (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60345 rocksdb:l ceph txt unknown /proc/59893/task/60345/exe (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60345 rocksdb:l ceph NOFD /proc/59893/task/60345/fd (opendir: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60346 rocksdb:l ceph cwd unknown /proc/59893/task/60346/cwd (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60346 rocksdb:l ceph rtd unknown /proc/59893/task/60346/root (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60346 rocksdb:l ceph txt unknown /proc/59893/task/60346/exe (readlink: Permission denied) 2024-03-16T08:19:37.815 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60346 rocksdb:l ceph NOFD /proc/59893/task/60346/fd (opendir: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60347 rocksdb:h ceph cwd unknown /proc/59893/task/60347/cwd (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60347 rocksdb:h ceph rtd unknown /proc/59893/task/60347/root (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60347 rocksdb:h ceph txt unknown /proc/59893/task/60347/exe (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60347 rocksdb:h ceph NOFD /proc/59893/task/60347/fd (opendir: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60746 bstore_ai ceph cwd unknown /proc/59893/task/60746/cwd (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60746 bstore_ai ceph rtd unknown /proc/59893/task/60746/root (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60746 bstore_ai ceph txt unknown /proc/59893/task/60746/exe (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60746 bstore_ai ceph NOFD /proc/59893/task/60746/fd (opendir: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60934 bstore_ai ceph cwd unknown /proc/59893/task/60934/cwd (readlink: Permission denied) 2024-03-16T08:19:37.816 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60934 bstore_ai ceph rtd unknown /proc/59893/task/60934/root (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60934 bstore_ai ceph txt unknown /proc/59893/task/60934/exe (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 60934 bstore_ai ceph NOFD /proc/59893/task/60934/fd (opendir: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61116 ceph-osd ceph cwd unknown /proc/59893/task/61116/cwd (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61116 ceph-osd ceph rtd unknown /proc/59893/task/61116/root (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61116 ceph-osd ceph txt unknown /proc/59893/task/61116/exe (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61116 ceph-osd ceph NOFD /proc/59893/task/61116/fd (opendir: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61117 cfin ceph cwd unknown /proc/59893/task/61117/cwd (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61117 cfin ceph rtd unknown /proc/59893/task/61117/root (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61117 cfin ceph txt unknown /proc/59893/task/61117/exe (readlink: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61117 cfin ceph NOFD /proc/59893/task/61117/fd (opendir: Permission denied) 2024-03-16T08:19:37.817 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61118 bstore_kv ceph cwd unknown /proc/59893/task/61118/cwd (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61118 bstore_kv ceph rtd unknown /proc/59893/task/61118/root (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61118 bstore_kv ceph txt unknown /proc/59893/task/61118/exe (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61118 bstore_kv ceph NOFD /proc/59893/task/61118/fd (opendir: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61119 bstore_kv ceph cwd unknown /proc/59893/task/61119/cwd (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61119 bstore_kv ceph rtd unknown /proc/59893/task/61119/root (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61119 bstore_kv ceph txt unknown /proc/59893/task/61119/exe (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61119 bstore_kv ceph NOFD /proc/59893/task/61119/fd (opendir: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61120 bstore_me ceph cwd unknown /proc/59893/task/61120/cwd (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61120 bstore_me ceph rtd unknown /proc/59893/task/61120/root (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61120 bstore_me ceph txt unknown /proc/59893/task/61120/exe (readlink: Permission denied) 2024-03-16T08:19:37.818 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61120 bstore_me ceph NOFD /proc/59893/task/61120/fd (opendir: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61121 ms_dispat ceph cwd unknown /proc/59893/task/61121/cwd (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61121 ms_dispat ceph rtd unknown /proc/59893/task/61121/root (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61121 ms_dispat ceph txt unknown /proc/59893/task/61121/exe (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61121 ms_dispat ceph NOFD /proc/59893/task/61121/fd (opendir: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61122 ms_local ceph cwd unknown /proc/59893/task/61122/cwd (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61122 ms_local ceph rtd unknown /proc/59893/task/61122/root (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61122 ms_local ceph txt unknown /proc/59893/task/61122/exe (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61122 ms_local ceph NOFD /proc/59893/task/61122/fd (opendir: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61123 safe_time ceph cwd unknown /proc/59893/task/61123/cwd (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61123 safe_time ceph rtd unknown /proc/59893/task/61123/root (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61123 safe_time ceph txt unknown /proc/59893/task/61123/exe (readlink: Permission denied) 2024-03-16T08:19:37.819 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61123 safe_time ceph NOFD /proc/59893/task/61123/fd (opendir: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61124 safe_time ceph cwd unknown /proc/59893/task/61124/cwd (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61124 safe_time ceph rtd unknown /proc/59893/task/61124/root (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61124 safe_time ceph txt unknown /proc/59893/task/61124/exe (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61124 safe_time ceph NOFD /proc/59893/task/61124/fd (opendir: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61125 ms_dispat ceph cwd unknown /proc/59893/task/61125/cwd (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61125 ms_dispat ceph rtd unknown /proc/59893/task/61125/root (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61125 ms_dispat ceph txt unknown /proc/59893/task/61125/exe (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61125 ms_dispat ceph NOFD /proc/59893/task/61125/fd (opendir: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61126 ms_local ceph cwd unknown /proc/59893/task/61126/cwd (readlink: Permission denied) 2024-03-16T08:19:37.820 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61126 ms_local ceph rtd unknown /proc/59893/task/61126/root (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61126 ms_local ceph txt unknown /proc/59893/task/61126/exe (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61126 ms_local ceph NOFD /proc/59893/task/61126/fd (opendir: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61127 ms_dispat ceph cwd unknown /proc/59893/task/61127/cwd (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61127 ms_dispat ceph rtd unknown /proc/59893/task/61127/root (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61127 ms_dispat ceph txt unknown /proc/59893/task/61127/exe (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61127 ms_dispat ceph NOFD /proc/59893/task/61127/fd (opendir: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61128 ms_local ceph cwd unknown /proc/59893/task/61128/cwd (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61128 ms_local ceph rtd unknown /proc/59893/task/61128/root (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61128 ms_local ceph txt unknown /proc/59893/task/61128/exe (readlink: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61128 ms_local ceph NOFD /proc/59893/task/61128/fd (opendir: Permission denied) 2024-03-16T08:19:37.821 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61129 ms_dispat ceph cwd unknown /proc/59893/task/61129/cwd (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61129 ms_dispat ceph rtd unknown /proc/59893/task/61129/root (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61129 ms_dispat ceph txt unknown /proc/59893/task/61129/exe (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61129 ms_dispat ceph NOFD /proc/59893/task/61129/fd (opendir: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61130 ms_local ceph cwd unknown /proc/59893/task/61130/cwd (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61130 ms_local ceph rtd unknown /proc/59893/task/61130/root (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61130 ms_local ceph txt unknown /proc/59893/task/61130/exe (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61130 ms_local ceph NOFD /proc/59893/task/61130/fd (opendir: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61131 ms_dispat ceph cwd unknown /proc/59893/task/61131/cwd (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61131 ms_dispat ceph rtd unknown /proc/59893/task/61131/root (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61131 ms_dispat ceph txt unknown /proc/59893/task/61131/exe (readlink: Permission denied) 2024-03-16T08:19:37.822 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61131 ms_dispat ceph NOFD /proc/59893/task/61131/fd (opendir: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61132 ms_local ceph cwd unknown /proc/59893/task/61132/cwd (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61132 ms_local ceph rtd unknown /proc/59893/task/61132/root (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61132 ms_local ceph txt unknown /proc/59893/task/61132/exe (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61132 ms_local ceph NOFD /proc/59893/task/61132/fd (opendir: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61133 ms_dispat ceph cwd unknown /proc/59893/task/61133/cwd (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61133 ms_dispat ceph rtd unknown /proc/59893/task/61133/root (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61133 ms_dispat ceph txt unknown /proc/59893/task/61133/exe (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61133 ms_dispat ceph NOFD /proc/59893/task/61133/fd (opendir: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61134 ms_local ceph cwd unknown /proc/59893/task/61134/cwd (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61134 ms_local ceph rtd unknown /proc/59893/task/61134/root (readlink: Permission denied) 2024-03-16T08:19:37.823 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61134 ms_local ceph txt unknown /proc/59893/task/61134/exe (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61134 ms_local ceph NOFD /proc/59893/task/61134/fd (opendir: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61135 ms_dispat ceph cwd unknown /proc/59893/task/61135/cwd (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61135 ms_dispat ceph rtd unknown /proc/59893/task/61135/root (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61135 ms_dispat ceph txt unknown /proc/59893/task/61135/exe (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61135 ms_dispat ceph NOFD /proc/59893/task/61135/fd (opendir: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61136 ms_local ceph cwd unknown /proc/59893/task/61136/cwd (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61136 ms_local ceph rtd unknown /proc/59893/task/61136/root (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61136 ms_local ceph txt unknown /proc/59893/task/61136/exe (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61136 ms_local ceph NOFD /proc/59893/task/61136/fd (opendir: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61137 fn_anonym ceph cwd unknown /proc/59893/task/61137/cwd (readlink: Permission denied) 2024-03-16T08:19:37.824 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61137 fn_anonym ceph rtd unknown /proc/59893/task/61137/root (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61137 fn_anonym ceph txt unknown /proc/59893/task/61137/exe (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61137 fn_anonym ceph NOFD /proc/59893/task/61137/fd (opendir: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61138 finisher ceph cwd unknown /proc/59893/task/61138/cwd (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61138 finisher ceph rtd unknown /proc/59893/task/61138/root (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61138 finisher ceph txt unknown /proc/59893/task/61138/exe (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61138 finisher ceph NOFD /proc/59893/task/61138/fd (opendir: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61139 safe_time ceph cwd unknown /proc/59893/task/61139/cwd (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61139 safe_time ceph rtd unknown /proc/59893/task/61139/root (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61139 safe_time ceph txt unknown /proc/59893/task/61139/exe (readlink: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61139 safe_time ceph NOFD /proc/59893/task/61139/fd (opendir: Permission denied) 2024-03-16T08:19:37.825 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61140 safe_time ceph cwd unknown /proc/59893/task/61140/cwd (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61140 safe_time ceph rtd unknown /proc/59893/task/61140/root (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61140 safe_time ceph txt unknown /proc/59893/task/61140/exe (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61140 safe_time ceph NOFD /proc/59893/task/61140/fd (opendir: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61141 ceph-osd ceph cwd unknown /proc/59893/task/61141/cwd (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61141 ceph-osd ceph rtd unknown /proc/59893/task/61141/root (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61141 ceph-osd ceph txt unknown /proc/59893/task/61141/exe (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61141 ceph-osd ceph NOFD /proc/59893/task/61141/fd (opendir: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61142 osd_srv_a ceph cwd unknown /proc/59893/task/61142/cwd (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61142 osd_srv_a ceph rtd unknown /proc/59893/task/61142/root (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61142 osd_srv_a ceph txt unknown /proc/59893/task/61142/exe (readlink: Permission denied) 2024-03-16T08:19:37.826 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61142 osd_srv_a ceph NOFD /proc/59893/task/61142/fd (opendir: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61143 tp_osd_tp ceph cwd unknown /proc/59893/task/61143/cwd (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61143 tp_osd_tp ceph rtd unknown /proc/59893/task/61143/root (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61143 tp_osd_tp ceph txt unknown /proc/59893/task/61143/exe (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61143 tp_osd_tp ceph NOFD /proc/59893/task/61143/fd (opendir: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61144 tp_osd_tp ceph cwd unknown /proc/59893/task/61144/cwd (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61144 tp_osd_tp ceph rtd unknown /proc/59893/task/61144/root (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61144 tp_osd_tp ceph txt unknown /proc/59893/task/61144/exe (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61144 tp_osd_tp ceph NOFD /proc/59893/task/61144/fd (opendir: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61145 tp_osd_tp ceph cwd unknown /proc/59893/task/61145/cwd (readlink: Permission denied) 2024-03-16T08:19:37.827 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61145 tp_osd_tp ceph rtd unknown /proc/59893/task/61145/root (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61145 tp_osd_tp ceph txt unknown /proc/59893/task/61145/exe (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61145 tp_osd_tp ceph NOFD /proc/59893/task/61145/fd (opendir: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61146 tp_osd_tp ceph cwd unknown /proc/59893/task/61146/cwd (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61146 tp_osd_tp ceph rtd unknown /proc/59893/task/61146/root (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61146 tp_osd_tp ceph txt unknown /proc/59893/task/61146/exe (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61146 tp_osd_tp ceph NOFD /proc/59893/task/61146/fd (opendir: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61147 tp_osd_tp ceph cwd unknown /proc/59893/task/61147/cwd (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61147 tp_osd_tp ceph rtd unknown /proc/59893/task/61147/root (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61147 tp_osd_tp ceph txt unknown /proc/59893/task/61147/exe (readlink: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61147 tp_osd_tp ceph NOFD /proc/59893/task/61147/fd (opendir: Permission denied) 2024-03-16T08:19:37.828 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61148 tp_osd_tp ceph cwd unknown /proc/59893/task/61148/cwd (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61148 tp_osd_tp ceph rtd unknown /proc/59893/task/61148/root (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61148 tp_osd_tp ceph txt unknown /proc/59893/task/61148/exe (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61148 tp_osd_tp ceph NOFD /proc/59893/task/61148/fd (opendir: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61149 tp_osd_tp ceph cwd unknown /proc/59893/task/61149/cwd (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61149 tp_osd_tp ceph rtd unknown /proc/59893/task/61149/root (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61149 tp_osd_tp ceph txt unknown /proc/59893/task/61149/exe (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61149 tp_osd_tp ceph NOFD /proc/59893/task/61149/fd (opendir: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61150 tp_osd_tp ceph cwd unknown /proc/59893/task/61150/cwd (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61150 tp_osd_tp ceph rtd unknown /proc/59893/task/61150/root (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61150 tp_osd_tp ceph txt unknown /proc/59893/task/61150/exe (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61150 tp_osd_tp ceph NOFD /proc/59893/task/61150/fd (opendir: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61151 tp_osd_tp ceph cwd unknown /proc/59893/task/61151/cwd (readlink: Permission denied) 2024-03-16T08:19:37.829 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61151 tp_osd_tp ceph rtd unknown /proc/59893/task/61151/root (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61151 tp_osd_tp ceph txt unknown /proc/59893/task/61151/exe (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61151 tp_osd_tp ceph NOFD /proc/59893/task/61151/fd (opendir: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61152 tp_osd_tp ceph cwd unknown /proc/59893/task/61152/cwd (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61152 tp_osd_tp ceph rtd unknown /proc/59893/task/61152/root (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61152 tp_osd_tp ceph txt unknown /proc/59893/task/61152/exe (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61152 tp_osd_tp ceph NOFD /proc/59893/task/61152/fd (opendir: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61153 tp_osd_tp ceph cwd unknown /proc/59893/task/61153/cwd (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61153 tp_osd_tp ceph rtd unknown /proc/59893/task/61153/root (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61153 tp_osd_tp ceph txt unknown /proc/59893/task/61153/exe (readlink: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61153 tp_osd_tp ceph NOFD /proc/59893/task/61153/fd (opendir: Permission denied) 2024-03-16T08:19:37.830 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61154 tp_osd_tp ceph cwd unknown /proc/59893/task/61154/cwd (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61154 tp_osd_tp ceph rtd unknown /proc/59893/task/61154/root (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61154 tp_osd_tp ceph txt unknown /proc/59893/task/61154/exe (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61154 tp_osd_tp ceph NOFD /proc/59893/task/61154/fd (opendir: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61155 tp_osd_tp ceph cwd unknown /proc/59893/task/61155/cwd (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61155 tp_osd_tp ceph rtd unknown /proc/59893/task/61155/root (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61155 tp_osd_tp ceph txt unknown /proc/59893/task/61155/exe (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61155 tp_osd_tp ceph NOFD /proc/59893/task/61155/fd (opendir: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61156 tp_osd_tp ceph cwd unknown /proc/59893/task/61156/cwd (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61156 tp_osd_tp ceph rtd unknown /proc/59893/task/61156/root (readlink: Permission denied) 2024-03-16T08:19:37.831 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61156 tp_osd_tp ceph txt unknown /proc/59893/task/61156/exe (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61156 tp_osd_tp ceph NOFD /proc/59893/task/61156/fd (opendir: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61157 tp_osd_tp ceph cwd unknown /proc/59893/task/61157/cwd (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61157 tp_osd_tp ceph rtd unknown /proc/59893/task/61157/root (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61157 tp_osd_tp ceph txt unknown /proc/59893/task/61157/exe (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61157 tp_osd_tp ceph NOFD /proc/59893/task/61157/fd (opendir: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61158 tp_osd_tp ceph cwd unknown /proc/59893/task/61158/cwd (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61158 tp_osd_tp ceph rtd unknown /proc/59893/task/61158/root (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61158 tp_osd_tp ceph txt unknown /proc/59893/task/61158/exe (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61158 tp_osd_tp ceph NOFD /proc/59893/task/61158/fd (opendir: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61159 osd_srv_h ceph cwd unknown /proc/59893/task/61159/cwd (readlink: Permission denied) 2024-03-16T08:19:37.832 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61159 osd_srv_h ceph rtd unknown /proc/59893/task/61159/root (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61159 osd_srv_h ceph txt unknown /proc/59893/task/61159/exe (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 59893 61159 osd_srv_h ceph NOFD /proc/59893/task/61159/fd (opendir: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 61160 root cwd unknown /proc/61160/cwd (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 61160 root rtd unknown /proc/61160/root (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 61160 root txt unknown /proc/61160/exe (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 61160 root NOFD /proc/61160/fd (opendir: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 61177 root cwd unknown /proc/61177/cwd (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 61177 root rtd unknown /proc/61177/root (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 61177 root txt unknown /proc/61177/exe (readlink: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 61177 root NOFD /proc/61177/fd (opendir: Permission denied) 2024-03-16T08:19:37.833 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 64145 root cwd unknown /proc/64145/cwd (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 64145 root rtd unknown /proc/64145/root (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 64145 root txt unknown /proc/64145/exe (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/5 64145 root NOFD /proc/64145/fd (opendir: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 64646 root cwd unknown /proc/64646/cwd (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 64646 root rtd unknown /proc/64646/root (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 64646 root txt unknown /proc/64646/exe (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/7 64646 root NOFD /proc/64646/fd (opendir: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66017 root cwd unknown /proc/66017/cwd (readlink: Permission denied) 2024-03-16T08:19:37.834 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66017 root rtd unknown /proc/66017/root (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66017 root txt unknown /proc/66017/exe (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66017 root NOFD /proc/66017/fd (opendir: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66018 root cwd unknown /proc/66018/cwd (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66018 root rtd unknown /proc/66018/root (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66018 root txt unknown /proc/66018/exe (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/2 66018 root NOFD /proc/66018/fd (opendir: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 66212 root cwd unknown /proc/66212/cwd (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 66212 root rtd unknown /proc/66212/root (readlink: Permission denied) 2024-03-16T08:19:37.835 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 66212 root txt unknown /proc/66212/exe (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 66212 root NOFD /proc/66212/fd (opendir: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 66516 root cwd unknown /proc/66516/cwd (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 66516 root rtd unknown /proc/66516/root (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 66516 root txt unknown /proc/66516/exe (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 66516 root NOFD /proc/66516/fd (opendir: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 66777 root cwd unknown /proc/66777/cwd (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 66777 root rtd unknown /proc/66777/root (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 66777 root txt unknown /proc/66777/exe (readlink: Permission denied) 2024-03-16T08:19:37.836 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 66777 root NOFD /proc/66777/fd (opendir: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 66779 root cwd unknown /proc/66779/cwd (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 66779 root rtd unknown /proc/66779/root (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 66779 root txt unknown /proc/66779/exe (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 66779 root NOFD /proc/66779/fd (opendir: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 ceph cwd unknown /proc/66781/cwd (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 ceph rtd unknown /proc/66781/root (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 ceph txt unknown /proc/66781/exe (readlink: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 ceph NOFD /proc/66781/fd (opendir: Permission denied) 2024-03-16T08:19:37.837 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66782 log ceph cwd unknown /proc/66781/task/66782/cwd (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66782 log ceph rtd unknown /proc/66781/task/66782/root (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66782 log ceph txt unknown /proc/66781/task/66782/exe (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66782 log ceph NOFD /proc/66781/task/66782/fd (opendir: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66784 msgr-work ceph cwd unknown /proc/66781/task/66784/cwd (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66784 msgr-work ceph rtd unknown /proc/66781/task/66784/root (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66784 msgr-work ceph txt unknown /proc/66781/task/66784/exe (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66784 msgr-work ceph NOFD /proc/66781/task/66784/fd (opendir: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66785 msgr-work ceph cwd unknown /proc/66781/task/66785/cwd (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66785 msgr-work ceph rtd unknown /proc/66781/task/66785/root (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66785 msgr-work ceph txt unknown /proc/66781/task/66785/exe (readlink: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66785 msgr-work ceph NOFD /proc/66781/task/66785/fd (opendir: Permission denied) 2024-03-16T08:19:37.838 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66786 msgr-work ceph cwd unknown /proc/66781/task/66786/cwd (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66786 msgr-work ceph rtd unknown /proc/66781/task/66786/root (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66786 msgr-work ceph txt unknown /proc/66781/task/66786/exe (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66786 msgr-work ceph NOFD /proc/66781/task/66786/fd (opendir: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66790 service ceph cwd unknown /proc/66781/task/66790/cwd (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66790 service ceph rtd unknown /proc/66781/task/66790/root (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66790 service ceph txt unknown /proc/66781/task/66790/exe (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66790 service ceph NOFD /proc/66781/task/66790/fd (opendir: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66791 admin_soc ceph cwd unknown /proc/66781/task/66791/cwd (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66791 admin_soc ceph rtd unknown /proc/66781/task/66791/root (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66791 admin_soc ceph txt unknown /proc/66781/task/66791/exe (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66791 admin_soc ceph NOFD /proc/66781/task/66791/fd (opendir: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66933 signal_ha ceph cwd unknown /proc/66781/task/66933/cwd (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66933 signal_ha ceph rtd unknown /proc/66781/task/66933/root (readlink: Permission denied) 2024-03-16T08:19:37.839 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66933 signal_ha ceph txt unknown /proc/66781/task/66933/exe (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66933 signal_ha ceph NOFD /proc/66781/task/66933/fd (opendir: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66934 io_contex ceph cwd unknown /proc/66781/task/66934/cwd (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66934 io_contex ceph rtd unknown /proc/66781/task/66934/root (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66934 io_contex ceph txt unknown /proc/66781/task/66934/exe (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66934 io_contex ceph NOFD /proc/66781/task/66934/fd (opendir: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66935 io_contex ceph cwd unknown /proc/66781/task/66935/cwd (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66935 io_contex ceph rtd unknown /proc/66781/task/66935/root (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66935 io_contex ceph txt unknown /proc/66781/task/66935/exe (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 66935 io_contex ceph NOFD /proc/66781/task/66935/fd (opendir: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67049 OpHistory ceph cwd unknown /proc/66781/task/67049/cwd (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67049 OpHistory ceph rtd unknown /proc/66781/task/67049/root (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67049 OpHistory ceph txt unknown /proc/66781/task/67049/exe (readlink: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67049 OpHistory ceph NOFD /proc/66781/task/67049/fd (opendir: Permission denied) 2024-03-16T08:19:37.840 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67050 ceph_time ceph cwd unknown /proc/66781/task/67050/cwd (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67050 ceph_time ceph rtd unknown /proc/66781/task/67050/root (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67050 ceph_time ceph txt unknown /proc/66781/task/67050/exe (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67050 ceph_time ceph NOFD /proc/66781/task/67050/fd (opendir: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67211 ceph-osd ceph cwd unknown /proc/66781/task/67211/cwd (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67211 ceph-osd ceph rtd unknown /proc/66781/task/67211/root (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67211 ceph-osd ceph txt unknown /proc/66781/task/67211/exe (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67211 ceph-osd ceph NOFD /proc/66781/task/67211/fd (opendir: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67212 safe_time ceph cwd unknown /proc/66781/task/67212/cwd (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67212 safe_time ceph rtd unknown /proc/66781/task/67212/root (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67212 safe_time ceph txt unknown /proc/66781/task/67212/exe (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67212 safe_time ceph NOFD /proc/66781/task/67212/fd (opendir: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67213 safe_time ceph cwd unknown /proc/66781/task/67213/cwd (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67213 safe_time ceph rtd unknown /proc/66781/task/67213/root (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67213 safe_time ceph txt unknown /proc/66781/task/67213/exe (readlink: Permission denied) 2024-03-16T08:19:37.841 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67213 safe_time ceph NOFD /proc/66781/task/67213/fd (opendir: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67214 safe_time ceph cwd unknown /proc/66781/task/67214/cwd (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67214 safe_time ceph rtd unknown /proc/66781/task/67214/root (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67214 safe_time ceph txt unknown /proc/66781/task/67214/exe (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67214 safe_time ceph NOFD /proc/66781/task/67214/fd (opendir: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67215 safe_time ceph cwd unknown /proc/66781/task/67215/cwd (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67215 safe_time ceph rtd unknown /proc/66781/task/67215/root (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67215 safe_time ceph txt unknown /proc/66781/task/67215/exe (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67215 safe_time ceph NOFD /proc/66781/task/67215/fd (opendir: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67216 fn_anonym ceph cwd unknown /proc/66781/task/67216/cwd (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67216 fn_anonym ceph rtd unknown /proc/66781/task/67216/root (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67216 fn_anonym ceph txt unknown /proc/66781/task/67216/exe (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67216 fn_anonym ceph NOFD /proc/66781/task/67216/fd (opendir: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67222 rocksdb:l ceph cwd unknown /proc/66781/task/67222/cwd (readlink: Permission denied) 2024-03-16T08:19:37.842 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67222 rocksdb:l ceph rtd unknown /proc/66781/task/67222/root (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67222 rocksdb:l ceph txt unknown /proc/66781/task/67222/exe (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67222 rocksdb:l ceph NOFD /proc/66781/task/67222/fd (opendir: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67223 rocksdb:l ceph cwd unknown /proc/66781/task/67223/cwd (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67223 rocksdb:l ceph rtd unknown /proc/66781/task/67223/root (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67223 rocksdb:l ceph txt unknown /proc/66781/task/67223/exe (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67223 rocksdb:l ceph NOFD /proc/66781/task/67223/fd (opendir: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67224 rocksdb:l ceph cwd unknown /proc/66781/task/67224/cwd (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67224 rocksdb:l ceph rtd unknown /proc/66781/task/67224/root (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67224 rocksdb:l ceph txt unknown /proc/66781/task/67224/exe (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67224 rocksdb:l ceph NOFD /proc/66781/task/67224/fd (opendir: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67225 rocksdb:h ceph cwd unknown /proc/66781/task/67225/cwd (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67225 rocksdb:h ceph rtd unknown /proc/66781/task/67225/root (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67225 rocksdb:h ceph txt unknown /proc/66781/task/67225/exe (readlink: Permission denied) 2024-03-16T08:19:37.843 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67225 rocksdb:h ceph NOFD /proc/66781/task/67225/fd (opendir: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67718 bstore_ai ceph cwd unknown /proc/66781/task/67718/cwd (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67718 bstore_ai ceph rtd unknown /proc/66781/task/67718/root (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67718 bstore_ai ceph txt unknown /proc/66781/task/67718/exe (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67718 bstore_ai ceph NOFD /proc/66781/task/67718/fd (opendir: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67903 bstore_ai ceph cwd unknown /proc/66781/task/67903/cwd (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67903 bstore_ai ceph rtd unknown /proc/66781/task/67903/root (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67903 bstore_ai ceph txt unknown /proc/66781/task/67903/exe (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 67903 bstore_ai ceph NOFD /proc/66781/task/67903/fd (opendir: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68085 ceph-osd ceph cwd unknown /proc/66781/task/68085/cwd (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68085 ceph-osd ceph rtd unknown /proc/66781/task/68085/root (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68085 ceph-osd ceph txt unknown /proc/66781/task/68085/exe (readlink: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68085 ceph-osd ceph NOFD /proc/66781/task/68085/fd (opendir: Permission denied) 2024-03-16T08:19:37.844 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68086 cfin ceph cwd unknown /proc/66781/task/68086/cwd (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68086 cfin ceph rtd unknown /proc/66781/task/68086/root (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68086 cfin ceph txt unknown /proc/66781/task/68086/exe (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68086 cfin ceph NOFD /proc/66781/task/68086/fd (opendir: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68087 bstore_kv ceph cwd unknown /proc/66781/task/68087/cwd (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68087 bstore_kv ceph rtd unknown /proc/66781/task/68087/root (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68087 bstore_kv ceph txt unknown /proc/66781/task/68087/exe (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68087 bstore_kv ceph NOFD /proc/66781/task/68087/fd (opendir: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68088 bstore_kv ceph cwd unknown /proc/66781/task/68088/cwd (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68088 bstore_kv ceph rtd unknown /proc/66781/task/68088/root (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68088 bstore_kv ceph txt unknown /proc/66781/task/68088/exe (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68088 bstore_kv ceph NOFD /proc/66781/task/68088/fd (opendir: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68089 bstore_me ceph cwd unknown /proc/66781/task/68089/cwd (readlink: Permission denied) 2024-03-16T08:19:37.845 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68089 bstore_me ceph rtd unknown /proc/66781/task/68089/root (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68089 bstore_me ceph txt unknown /proc/66781/task/68089/exe (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68089 bstore_me ceph NOFD /proc/66781/task/68089/fd (opendir: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68090 ms_dispat ceph cwd unknown /proc/66781/task/68090/cwd (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68090 ms_dispat ceph rtd unknown /proc/66781/task/68090/root (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68090 ms_dispat ceph txt unknown /proc/66781/task/68090/exe (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68090 ms_dispat ceph NOFD /proc/66781/task/68090/fd (opendir: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68091 ms_local ceph cwd unknown /proc/66781/task/68091/cwd (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68091 ms_local ceph rtd unknown /proc/66781/task/68091/root (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68091 ms_local ceph txt unknown /proc/66781/task/68091/exe (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68091 ms_local ceph NOFD /proc/66781/task/68091/fd (opendir: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68092 safe_time ceph cwd unknown /proc/66781/task/68092/cwd (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68092 safe_time ceph rtd unknown /proc/66781/task/68092/root (readlink: Permission denied) 2024-03-16T08:19:37.846 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68092 safe_time ceph txt unknown /proc/66781/task/68092/exe (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68092 safe_time ceph NOFD /proc/66781/task/68092/fd (opendir: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68093 safe_time ceph cwd unknown /proc/66781/task/68093/cwd (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68093 safe_time ceph rtd unknown /proc/66781/task/68093/root (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68093 safe_time ceph txt unknown /proc/66781/task/68093/exe (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68093 safe_time ceph NOFD /proc/66781/task/68093/fd (opendir: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68094 ms_dispat ceph cwd unknown /proc/66781/task/68094/cwd (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68094 ms_dispat ceph rtd unknown /proc/66781/task/68094/root (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68094 ms_dispat ceph txt unknown /proc/66781/task/68094/exe (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68094 ms_dispat ceph NOFD /proc/66781/task/68094/fd (opendir: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68095 ms_local ceph cwd unknown /proc/66781/task/68095/cwd (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68095 ms_local ceph rtd unknown /proc/66781/task/68095/root (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68095 ms_local ceph txt unknown /proc/66781/task/68095/exe (readlink: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68095 ms_local ceph NOFD /proc/66781/task/68095/fd (opendir: Permission denied) 2024-03-16T08:19:37.847 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68096 ms_dispat ceph cwd unknown /proc/66781/task/68096/cwd (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68096 ms_dispat ceph rtd unknown /proc/66781/task/68096/root (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68096 ms_dispat ceph txt unknown /proc/66781/task/68096/exe (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68096 ms_dispat ceph NOFD /proc/66781/task/68096/fd (opendir: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68097 ms_local ceph cwd unknown /proc/66781/task/68097/cwd (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68097 ms_local ceph rtd unknown /proc/66781/task/68097/root (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68097 ms_local ceph txt unknown /proc/66781/task/68097/exe (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68097 ms_local ceph NOFD /proc/66781/task/68097/fd (opendir: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68098 ms_dispat ceph cwd unknown /proc/66781/task/68098/cwd (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68098 ms_dispat ceph rtd unknown /proc/66781/task/68098/root (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68098 ms_dispat ceph txt unknown /proc/66781/task/68098/exe (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68098 ms_dispat ceph NOFD /proc/66781/task/68098/fd (opendir: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68099 ms_local ceph cwd unknown /proc/66781/task/68099/cwd (readlink: Permission denied) 2024-03-16T08:19:37.848 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68099 ms_local ceph rtd unknown /proc/66781/task/68099/root (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68099 ms_local ceph txt unknown /proc/66781/task/68099/exe (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68099 ms_local ceph NOFD /proc/66781/task/68099/fd (opendir: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68100 ms_dispat ceph cwd unknown /proc/66781/task/68100/cwd (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68100 ms_dispat ceph rtd unknown /proc/66781/task/68100/root (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68100 ms_dispat ceph txt unknown /proc/66781/task/68100/exe (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68100 ms_dispat ceph NOFD /proc/66781/task/68100/fd (opendir: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68101 ms_local ceph cwd unknown /proc/66781/task/68101/cwd (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68101 ms_local ceph rtd unknown /proc/66781/task/68101/root (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68101 ms_local ceph txt unknown /proc/66781/task/68101/exe (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68101 ms_local ceph NOFD /proc/66781/task/68101/fd (opendir: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68102 ms_dispat ceph cwd unknown /proc/66781/task/68102/cwd (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68102 ms_dispat ceph rtd unknown /proc/66781/task/68102/root (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68102 ms_dispat ceph txt unknown /proc/66781/task/68102/exe (readlink: Permission denied) 2024-03-16T08:19:37.849 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68102 ms_dispat ceph NOFD /proc/66781/task/68102/fd (opendir: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68103 ms_local ceph cwd unknown /proc/66781/task/68103/cwd (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68103 ms_local ceph rtd unknown /proc/66781/task/68103/root (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68103 ms_local ceph txt unknown /proc/66781/task/68103/exe (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68103 ms_local ceph NOFD /proc/66781/task/68103/fd (opendir: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68104 ms_dispat ceph cwd unknown /proc/66781/task/68104/cwd (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68104 ms_dispat ceph rtd unknown /proc/66781/task/68104/root (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68104 ms_dispat ceph txt unknown /proc/66781/task/68104/exe (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68104 ms_dispat ceph NOFD /proc/66781/task/68104/fd (opendir: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68105 ms_local ceph cwd unknown /proc/66781/task/68105/cwd (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68105 ms_local ceph rtd unknown /proc/66781/task/68105/root (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68105 ms_local ceph txt unknown /proc/66781/task/68105/exe (readlink: Permission denied) 2024-03-16T08:19:37.850 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68105 ms_local ceph NOFD /proc/66781/task/68105/fd (opendir: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68106 fn_anonym ceph cwd unknown /proc/66781/task/68106/cwd (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68106 fn_anonym ceph rtd unknown /proc/66781/task/68106/root (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68106 fn_anonym ceph txt unknown /proc/66781/task/68106/exe (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68106 fn_anonym ceph NOFD /proc/66781/task/68106/fd (opendir: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68107 finisher ceph cwd unknown /proc/66781/task/68107/cwd (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68107 finisher ceph rtd unknown /proc/66781/task/68107/root (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68107 finisher ceph txt unknown /proc/66781/task/68107/exe (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68107 finisher ceph NOFD /proc/66781/task/68107/fd (opendir: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68108 safe_time ceph cwd unknown /proc/66781/task/68108/cwd (readlink: Permission denied) 2024-03-16T08:19:37.851 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68108 safe_time ceph rtd unknown /proc/66781/task/68108/root (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68108 safe_time ceph txt unknown /proc/66781/task/68108/exe (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68108 safe_time ceph NOFD /proc/66781/task/68108/fd (opendir: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68109 safe_time ceph cwd unknown /proc/66781/task/68109/cwd (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68109 safe_time ceph rtd unknown /proc/66781/task/68109/root (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68109 safe_time ceph txt unknown /proc/66781/task/68109/exe (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68109 safe_time ceph NOFD /proc/66781/task/68109/fd (opendir: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68110 ceph-osd ceph cwd unknown /proc/66781/task/68110/cwd (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68110 ceph-osd ceph rtd unknown /proc/66781/task/68110/root (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68110 ceph-osd ceph txt unknown /proc/66781/task/68110/exe (readlink: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68110 ceph-osd ceph NOFD /proc/66781/task/68110/fd (opendir: Permission denied) 2024-03-16T08:19:37.852 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68111 osd_srv_a ceph cwd unknown /proc/66781/task/68111/cwd (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68111 osd_srv_a ceph rtd unknown /proc/66781/task/68111/root (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68111 osd_srv_a ceph txt unknown /proc/66781/task/68111/exe (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68111 osd_srv_a ceph NOFD /proc/66781/task/68111/fd (opendir: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68112 tp_osd_tp ceph cwd unknown /proc/66781/task/68112/cwd (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68112 tp_osd_tp ceph rtd unknown /proc/66781/task/68112/root (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68112 tp_osd_tp ceph txt unknown /proc/66781/task/68112/exe (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68112 tp_osd_tp ceph NOFD /proc/66781/task/68112/fd (opendir: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68113 tp_osd_tp ceph cwd unknown /proc/66781/task/68113/cwd (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68113 tp_osd_tp ceph rtd unknown /proc/66781/task/68113/root (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68113 tp_osd_tp ceph txt unknown /proc/66781/task/68113/exe (readlink: Permission denied) 2024-03-16T08:19:37.853 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68113 tp_osd_tp ceph NOFD /proc/66781/task/68113/fd (opendir: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68114 tp_osd_tp ceph cwd unknown /proc/66781/task/68114/cwd (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68114 tp_osd_tp ceph rtd unknown /proc/66781/task/68114/root (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68114 tp_osd_tp ceph txt unknown /proc/66781/task/68114/exe (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68114 tp_osd_tp ceph NOFD /proc/66781/task/68114/fd (opendir: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68115 tp_osd_tp ceph cwd unknown /proc/66781/task/68115/cwd (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68115 tp_osd_tp ceph rtd unknown /proc/66781/task/68115/root (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68115 tp_osd_tp ceph txt unknown /proc/66781/task/68115/exe (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68115 tp_osd_tp ceph NOFD /proc/66781/task/68115/fd (opendir: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68116 tp_osd_tp ceph cwd unknown /proc/66781/task/68116/cwd (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68116 tp_osd_tp ceph rtd unknown /proc/66781/task/68116/root (readlink: Permission denied) 2024-03-16T08:19:37.854 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68116 tp_osd_tp ceph txt unknown /proc/66781/task/68116/exe (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68116 tp_osd_tp ceph NOFD /proc/66781/task/68116/fd (opendir: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68117 tp_osd_tp ceph cwd unknown /proc/66781/task/68117/cwd (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68117 tp_osd_tp ceph rtd unknown /proc/66781/task/68117/root (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68117 tp_osd_tp ceph txt unknown /proc/66781/task/68117/exe (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68117 tp_osd_tp ceph NOFD /proc/66781/task/68117/fd (opendir: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68118 tp_osd_tp ceph cwd unknown /proc/66781/task/68118/cwd (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68118 tp_osd_tp ceph rtd unknown /proc/66781/task/68118/root (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68118 tp_osd_tp ceph txt unknown /proc/66781/task/68118/exe (readlink: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68118 tp_osd_tp ceph NOFD /proc/66781/task/68118/fd (opendir: Permission denied) 2024-03-16T08:19:37.855 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68119 tp_osd_tp ceph cwd unknown /proc/66781/task/68119/cwd (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68119 tp_osd_tp ceph rtd unknown /proc/66781/task/68119/root (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68119 tp_osd_tp ceph txt unknown /proc/66781/task/68119/exe (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68119 tp_osd_tp ceph NOFD /proc/66781/task/68119/fd (opendir: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68120 tp_osd_tp ceph cwd unknown /proc/66781/task/68120/cwd (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68120 tp_osd_tp ceph rtd unknown /proc/66781/task/68120/root (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68120 tp_osd_tp ceph txt unknown /proc/66781/task/68120/exe (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68120 tp_osd_tp ceph NOFD /proc/66781/task/68120/fd (opendir: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68121 tp_osd_tp ceph cwd unknown /proc/66781/task/68121/cwd (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68121 tp_osd_tp ceph rtd unknown /proc/66781/task/68121/root (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68121 tp_osd_tp ceph txt unknown /proc/66781/task/68121/exe (readlink: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68121 tp_osd_tp ceph NOFD /proc/66781/task/68121/fd (opendir: Permission denied) 2024-03-16T08:19:37.856 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68122 tp_osd_tp ceph cwd unknown /proc/66781/task/68122/cwd (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68122 tp_osd_tp ceph rtd unknown /proc/66781/task/68122/root (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68122 tp_osd_tp ceph txt unknown /proc/66781/task/68122/exe (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68122 tp_osd_tp ceph NOFD /proc/66781/task/68122/fd (opendir: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68123 tp_osd_tp ceph cwd unknown /proc/66781/task/68123/cwd (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68123 tp_osd_tp ceph rtd unknown /proc/66781/task/68123/root (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68123 tp_osd_tp ceph txt unknown /proc/66781/task/68123/exe (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68123 tp_osd_tp ceph NOFD /proc/66781/task/68123/fd (opendir: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68124 tp_osd_tp ceph cwd unknown /proc/66781/task/68124/cwd (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68124 tp_osd_tp ceph rtd unknown /proc/66781/task/68124/root (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68124 tp_osd_tp ceph txt unknown /proc/66781/task/68124/exe (readlink: Permission denied) 2024-03-16T08:19:37.857 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68124 tp_osd_tp ceph NOFD /proc/66781/task/68124/fd (opendir: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68125 tp_osd_tp ceph cwd unknown /proc/66781/task/68125/cwd (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68125 tp_osd_tp ceph rtd unknown /proc/66781/task/68125/root (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68125 tp_osd_tp ceph txt unknown /proc/66781/task/68125/exe (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68125 tp_osd_tp ceph NOFD /proc/66781/task/68125/fd (opendir: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68126 tp_osd_tp ceph cwd unknown /proc/66781/task/68126/cwd (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68126 tp_osd_tp ceph rtd unknown /proc/66781/task/68126/root (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68126 tp_osd_tp ceph txt unknown /proc/66781/task/68126/exe (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68126 tp_osd_tp ceph NOFD /proc/66781/task/68126/fd (opendir: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68127 tp_osd_tp ceph cwd unknown /proc/66781/task/68127/cwd (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68127 tp_osd_tp ceph rtd unknown /proc/66781/task/68127/root (readlink: Permission denied) 2024-03-16T08:19:37.858 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68127 tp_osd_tp ceph txt unknown /proc/66781/task/68127/exe (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68127 tp_osd_tp ceph NOFD /proc/66781/task/68127/fd (opendir: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68128 osd_srv_h ceph cwd unknown /proc/66781/task/68128/cwd (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68128 osd_srv_h ceph rtd unknown /proc/66781/task/68128/root (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68128 osd_srv_h ceph txt unknown /proc/66781/task/68128/exe (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-osd 66781 68128 osd_srv_h ceph NOFD /proc/66781/task/68128/fd (opendir: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 68222 root cwd unknown /proc/68222/cwd (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 68222 root rtd unknown /proc/68222/root (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 68222 root txt unknown /proc/68222/exe (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 68222 root NOFD /proc/68222/fd (opendir: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 68253 root cwd unknown /proc/68253/cwd (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 68253 root rtd unknown /proc/68253/root (readlink: Permission denied) 2024-03-16T08:19:37.859 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 68253 root txt unknown /proc/68253/exe (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 68253 root NOFD /proc/68253/fd (opendir: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 70739 root cwd unknown /proc/70739/cwd (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 70739 root rtd unknown /proc/70739/root (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 70739 root txt unknown /proc/70739/exe (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/3 70739 root NOFD /proc/70739/fd (opendir: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71425 root cwd unknown /proc/71425/cwd (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71425 root rtd unknown /proc/71425/root (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71425 root txt unknown /proc/71425/exe (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71425 root NOFD /proc/71425/fd (opendir: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71442 root cwd unknown /proc/71442/cwd (readlink: Permission denied) 2024-03-16T08:19:37.860 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71442 root rtd unknown /proc/71442/root (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71442 root txt unknown /proc/71442/exe (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71442 root NOFD /proc/71442/fd (opendir: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71444 root cwd unknown /proc/71444/cwd (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71444 root rtd unknown /proc/71444/root (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71444 root txt unknown /proc/71444/exe (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71444 root NOFD /proc/71444/fd (opendir: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71460 root cwd unknown /proc/71460/cwd (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71460 root rtd unknown /proc/71460/root (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71460 root txt unknown /proc/71460/exe (readlink: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71460 root NOFD /proc/71460/fd (opendir: Permission denied) 2024-03-16T08:19:37.861 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71461 root cwd unknown /proc/71461/cwd (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71461 root rtd unknown /proc/71461/root (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71461 root txt unknown /proc/71461/exe (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71461 root NOFD /proc/71461/fd (opendir: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71478 root cwd unknown /proc/71478/cwd (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71478 root rtd unknown /proc/71478/root (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71478 root txt unknown /proc/71478/exe (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71478 root NOFD /proc/71478/fd (opendir: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71479 root cwd unknown /proc/71479/cwd (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71479 root rtd unknown /proc/71479/root (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71479 root txt unknown /proc/71479/exe (readlink: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 71479 root NOFD /proc/71479/fd (opendir: Permission denied) 2024-03-16T08:19:37.862 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71496 root cwd unknown /proc/71496/cwd (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71496 root rtd unknown /proc/71496/root (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71496 root txt unknown /proc/71496/exe (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:journalct 71496 root NOFD /proc/71496/fd (opendir: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72020 root cwd unknown /proc/72020/cwd (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72020 root rtd unknown /proc/72020/root (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72020 root txt unknown /proc/72020/exe (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72020 root NOFD /proc/72020/fd (opendir: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72022 root cwd unknown /proc/72022/cwd (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72022 root rtd unknown /proc/72022/root (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72022 root txt unknown /proc/72022/exe (readlink: Permission denied) 2024-03-16T08:19:37.863 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72022 root NOFD /proc/72022/fd (opendir: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 ceph cwd unknown /proc/72024/cwd (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 ceph rtd unknown /proc/72024/root (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 ceph txt unknown /proc/72024/exe (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 ceph NOFD /proc/72024/fd (opendir: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72025 log ceph cwd unknown /proc/72024/task/72025/cwd (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72025 log ceph rtd unknown /proc/72024/task/72025/root (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72025 log ceph txt unknown /proc/72024/task/72025/exe (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72025 log ceph NOFD /proc/72024/task/72025/fd (opendir: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72027 msgr-work ceph cwd unknown /proc/72024/task/72027/cwd (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72027 msgr-work ceph rtd unknown /proc/72024/task/72027/root (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72027 msgr-work ceph txt unknown /proc/72024/task/72027/exe (readlink: Permission denied) 2024-03-16T08:19:37.864 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72027 msgr-work ceph NOFD /proc/72024/task/72027/fd (opendir: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72028 msgr-work ceph cwd unknown /proc/72024/task/72028/cwd (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72028 msgr-work ceph rtd unknown /proc/72024/task/72028/root (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72028 msgr-work ceph txt unknown /proc/72024/task/72028/exe (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72028 msgr-work ceph NOFD /proc/72024/task/72028/fd (opendir: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72029 msgr-work ceph cwd unknown /proc/72024/task/72029/cwd (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72029 msgr-work ceph rtd unknown /proc/72024/task/72029/root (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72029 msgr-work ceph txt unknown /proc/72024/task/72029/exe (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72029 msgr-work ceph NOFD /proc/72024/task/72029/fd (opendir: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72033 service ceph cwd unknown /proc/72024/task/72033/cwd (readlink: Permission denied) 2024-03-16T08:19:37.865 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72033 service ceph rtd unknown /proc/72024/task/72033/root (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72033 service ceph txt unknown /proc/72024/task/72033/exe (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72033 service ceph NOFD /proc/72024/task/72033/fd (opendir: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72034 admin_soc ceph cwd unknown /proc/72024/task/72034/cwd (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72034 admin_soc ceph rtd unknown /proc/72024/task/72034/root (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72034 admin_soc ceph txt unknown /proc/72024/task/72034/exe (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72034 admin_soc ceph NOFD /proc/72024/task/72034/fd (opendir: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72035 signal_ha ceph cwd unknown /proc/72024/task/72035/cwd (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72035 signal_ha ceph rtd unknown /proc/72024/task/72035/root (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72035 signal_ha ceph txt unknown /proc/72024/task/72035/exe (readlink: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72035 signal_ha ceph NOFD /proc/72024/task/72035/fd (opendir: Permission denied) 2024-03-16T08:19:37.866 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72036 io_contex ceph cwd unknown /proc/72024/task/72036/cwd (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72036 io_contex ceph rtd unknown /proc/72024/task/72036/root (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72036 io_contex ceph txt unknown /proc/72024/task/72036/exe (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72036 io_contex ceph NOFD /proc/72024/task/72036/fd (opendir: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72037 io_contex ceph cwd unknown /proc/72024/task/72037/cwd (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72037 io_contex ceph rtd unknown /proc/72024/task/72037/root (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72037 io_contex ceph txt unknown /proc/72024/task/72037/exe (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72037 io_contex ceph NOFD /proc/72024/task/72037/fd (opendir: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72038 ms_dispat ceph cwd unknown /proc/72024/task/72038/cwd (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72038 ms_dispat ceph rtd unknown /proc/72024/task/72038/root (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72038 ms_dispat ceph txt unknown /proc/72024/task/72038/exe (readlink: Permission denied) 2024-03-16T08:19:37.867 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72038 ms_dispat ceph NOFD /proc/72024/task/72038/fd (opendir: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72039 ms_local ceph cwd unknown /proc/72024/task/72039/cwd (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72039 ms_local ceph rtd unknown /proc/72024/task/72039/root (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72039 ms_local ceph txt unknown /proc/72024/task/72039/exe (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72039 ms_local ceph NOFD /proc/72024/task/72039/fd (opendir: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72040 safe_time ceph cwd unknown /proc/72024/task/72040/cwd (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72040 safe_time ceph rtd unknown /proc/72024/task/72040/root (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72040 safe_time ceph txt unknown /proc/72024/task/72040/exe (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72040 safe_time ceph NOFD /proc/72024/task/72040/fd (opendir: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72041 safe_time ceph cwd unknown /proc/72024/task/72041/cwd (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72041 safe_time ceph rtd unknown /proc/72024/task/72041/root (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72041 safe_time ceph txt unknown /proc/72024/task/72041/exe (readlink: Permission denied) 2024-03-16T08:19:37.868 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72041 safe_time ceph NOFD /proc/72024/task/72041/fd (opendir: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72042 ceph-mds ceph cwd unknown /proc/72024/task/72042/cwd (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72042 ceph-mds ceph rtd unknown /proc/72024/task/72042/root (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72042 ceph-mds ceph txt unknown /proc/72024/task/72042/exe (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72042 ceph-mds ceph NOFD /proc/72024/task/72042/fd (opendir: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72115 safe_time ceph cwd unknown /proc/72024/task/72115/cwd (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72115 safe_time ceph rtd unknown /proc/72024/task/72115/root (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72115 safe_time ceph txt unknown /proc/72024/task/72115/exe (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72024 72115 safe_time ceph NOFD /proc/72024/task/72115/fd (opendir: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72374 root cwd unknown /proc/72374/cwd (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72374 root rtd unknown /proc/72374/root (readlink: Permission denied) 2024-03-16T08:19:37.869 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72374 root txt unknown /proc/72374/exe (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72374 root NOFD /proc/72374/fd (opendir: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72376 root cwd unknown /proc/72376/cwd (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72376 root rtd unknown /proc/72376/root (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72376 root txt unknown /proc/72376/exe (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72376 root NOFD /proc/72376/fd (opendir: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 ceph cwd unknown /proc/72378/cwd (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 ceph rtd unknown /proc/72378/root (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 ceph txt unknown /proc/72378/exe (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 ceph NOFD /proc/72378/fd (opendir: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72381 log ceph cwd unknown /proc/72378/task/72381/cwd (readlink: Permission denied) 2024-03-16T08:19:37.870 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72381 log ceph rtd unknown /proc/72378/task/72381/root (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72381 log ceph txt unknown /proc/72378/task/72381/exe (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72381 log ceph NOFD /proc/72378/task/72381/fd (opendir: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72383 msgr-work ceph cwd unknown /proc/72378/task/72383/cwd (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72383 msgr-work ceph rtd unknown /proc/72378/task/72383/root (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72383 msgr-work ceph txt unknown /proc/72378/task/72383/exe (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72383 msgr-work ceph NOFD /proc/72378/task/72383/fd (opendir: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72384 msgr-work ceph cwd unknown /proc/72378/task/72384/cwd (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72384 msgr-work ceph rtd unknown /proc/72378/task/72384/root (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72384 msgr-work ceph txt unknown /proc/72378/task/72384/exe (readlink: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72384 msgr-work ceph NOFD /proc/72378/task/72384/fd (opendir: Permission denied) 2024-03-16T08:19:37.871 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72385 msgr-work ceph cwd unknown /proc/72378/task/72385/cwd (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72385 msgr-work ceph rtd unknown /proc/72378/task/72385/root (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72385 msgr-work ceph txt unknown /proc/72378/task/72385/exe (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72385 msgr-work ceph NOFD /proc/72378/task/72385/fd (opendir: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72389 service ceph cwd unknown /proc/72378/task/72389/cwd (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72389 service ceph rtd unknown /proc/72378/task/72389/root (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72389 service ceph txt unknown /proc/72378/task/72389/exe (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72389 service ceph NOFD /proc/72378/task/72389/fd (opendir: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72390 admin_soc ceph cwd unknown /proc/72378/task/72390/cwd (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72390 admin_soc ceph rtd unknown /proc/72378/task/72390/root (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72390 admin_soc ceph txt unknown /proc/72378/task/72390/exe (readlink: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72390 admin_soc ceph NOFD /proc/72378/task/72390/fd (opendir: Permission denied) 2024-03-16T08:19:37.872 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72391 signal_ha ceph cwd unknown /proc/72378/task/72391/cwd (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72391 signal_ha ceph rtd unknown /proc/72378/task/72391/root (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72391 signal_ha ceph txt unknown /proc/72378/task/72391/exe (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72391 signal_ha ceph NOFD /proc/72378/task/72391/fd (opendir: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72392 io_contex ceph cwd unknown /proc/72378/task/72392/cwd (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72392 io_contex ceph rtd unknown /proc/72378/task/72392/root (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72392 io_contex ceph txt unknown /proc/72378/task/72392/exe (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72392 io_contex ceph NOFD /proc/72378/task/72392/fd (opendir: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72393 io_contex ceph cwd unknown /proc/72378/task/72393/cwd (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72393 io_contex ceph rtd unknown /proc/72378/task/72393/root (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72393 io_contex ceph txt unknown /proc/72378/task/72393/exe (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72393 io_contex ceph NOFD /proc/72378/task/72393/fd (opendir: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72394 ms_dispat ceph cwd unknown /proc/72378/task/72394/cwd (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72394 ms_dispat ceph rtd unknown /proc/72378/task/72394/root (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72394 ms_dispat ceph txt unknown /proc/72378/task/72394/exe (readlink: Permission denied) 2024-03-16T08:19:37.873 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72394 ms_dispat ceph NOFD /proc/72378/task/72394/fd (opendir: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72395 ms_local ceph cwd unknown /proc/72378/task/72395/cwd (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72395 ms_local ceph rtd unknown /proc/72378/task/72395/root (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72395 ms_local ceph txt unknown /proc/72378/task/72395/exe (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72395 ms_local ceph NOFD /proc/72378/task/72395/fd (opendir: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72396 safe_time ceph cwd unknown /proc/72378/task/72396/cwd (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72396 safe_time ceph rtd unknown /proc/72378/task/72396/root (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72396 safe_time ceph txt unknown /proc/72378/task/72396/exe (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72396 safe_time ceph NOFD /proc/72378/task/72396/fd (opendir: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72398 safe_time ceph cwd unknown /proc/72378/task/72398/cwd (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72398 safe_time ceph rtd unknown /proc/72378/task/72398/root (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72398 safe_time ceph txt unknown /proc/72378/task/72398/exe (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72398 safe_time ceph NOFD /proc/72378/task/72398/fd (opendir: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72399 ceph-mds ceph cwd unknown /proc/72378/task/72399/cwd (readlink: Permission denied) 2024-03-16T08:19:37.874 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72399 ceph-mds ceph rtd unknown /proc/72378/task/72399/root (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72399 ceph-mds ceph txt unknown /proc/72378/task/72399/exe (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72399 ceph-mds ceph NOFD /proc/72378/task/72399/fd (opendir: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72411 safe_time ceph cwd unknown /proc/72378/task/72411/cwd (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72411 safe_time ceph rtd unknown /proc/72378/task/72411/root (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72411 safe_time ceph txt unknown /proc/72378/task/72411/exe (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72378 72411 safe_time ceph NOFD /proc/72378/task/72411/fd (opendir: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72979 root cwd unknown /proc/72979/cwd (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72979 root rtd unknown /proc/72979/root (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72979 root txt unknown /proc/72979/exe (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 72979 root NOFD /proc/72979/fd (opendir: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72981 root cwd unknown /proc/72981/cwd (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72981 root rtd unknown /proc/72981/root (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72981 root txt unknown /proc/72981/exe (readlink: Permission denied) 2024-03-16T08:19:37.875 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 72981 root NOFD /proc/72981/fd (opendir: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 ceph cwd unknown /proc/72983/cwd (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 ceph rtd unknown /proc/72983/root (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 ceph txt unknown /proc/72983/exe (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 ceph NOFD /proc/72983/fd (opendir: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72984 log ceph cwd unknown /proc/72983/task/72984/cwd (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72984 log ceph rtd unknown /proc/72983/task/72984/root (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72984 log ceph txt unknown /proc/72983/task/72984/exe (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72984 log ceph NOFD /proc/72983/task/72984/fd (opendir: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72986 msgr-work ceph cwd unknown /proc/72983/task/72986/cwd (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72986 msgr-work ceph rtd unknown /proc/72983/task/72986/root (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72986 msgr-work ceph txt unknown /proc/72983/task/72986/exe (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72986 msgr-work ceph NOFD /proc/72983/task/72986/fd (opendir: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72987 msgr-work ceph cwd unknown /proc/72983/task/72987/cwd (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72987 msgr-work ceph rtd unknown /proc/72983/task/72987/root (readlink: Permission denied) 2024-03-16T08:19:37.876 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72987 msgr-work ceph txt unknown /proc/72983/task/72987/exe (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72987 msgr-work ceph NOFD /proc/72983/task/72987/fd (opendir: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72988 msgr-work ceph cwd unknown /proc/72983/task/72988/cwd (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72988 msgr-work ceph rtd unknown /proc/72983/task/72988/root (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72988 msgr-work ceph txt unknown /proc/72983/task/72988/exe (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72988 msgr-work ceph NOFD /proc/72983/task/72988/fd (opendir: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72992 service ceph cwd unknown /proc/72983/task/72992/cwd (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72992 service ceph rtd unknown /proc/72983/task/72992/root (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72992 service ceph txt unknown /proc/72983/task/72992/exe (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72992 service ceph NOFD /proc/72983/task/72992/fd (opendir: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72993 admin_soc ceph cwd unknown /proc/72983/task/72993/cwd (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72993 admin_soc ceph rtd unknown /proc/72983/task/72993/root (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72993 admin_soc ceph txt unknown /proc/72983/task/72993/exe (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72993 admin_soc ceph NOFD /proc/72983/task/72993/fd (opendir: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72994 signal_ha ceph cwd unknown /proc/72983/task/72994/cwd (readlink: Permission denied) 2024-03-16T08:19:37.877 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72994 signal_ha ceph rtd unknown /proc/72983/task/72994/root (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72994 signal_ha ceph txt unknown /proc/72983/task/72994/exe (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72994 signal_ha ceph NOFD /proc/72983/task/72994/fd (opendir: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72995 io_contex ceph cwd unknown /proc/72983/task/72995/cwd (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72995 io_contex ceph rtd unknown /proc/72983/task/72995/root (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72995 io_contex ceph txt unknown /proc/72983/task/72995/exe (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72995 io_contex ceph NOFD /proc/72983/task/72995/fd (opendir: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72996 io_contex ceph cwd unknown /proc/72983/task/72996/cwd (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72996 io_contex ceph rtd unknown /proc/72983/task/72996/root (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72996 io_contex ceph txt unknown /proc/72983/task/72996/exe (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72996 io_contex ceph NOFD /proc/72983/task/72996/fd (opendir: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72997 ms_dispat ceph cwd unknown /proc/72983/task/72997/cwd (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72997 ms_dispat ceph rtd unknown /proc/72983/task/72997/root (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72997 ms_dispat ceph txt unknown /proc/72983/task/72997/exe (readlink: Permission denied) 2024-03-16T08:19:37.878 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72997 ms_dispat ceph NOFD /proc/72983/task/72997/fd (opendir: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72998 ms_local ceph cwd unknown /proc/72983/task/72998/cwd (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72998 ms_local ceph rtd unknown /proc/72983/task/72998/root (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72998 ms_local ceph txt unknown /proc/72983/task/72998/exe (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72998 ms_local ceph NOFD /proc/72983/task/72998/fd (opendir: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72999 safe_time ceph cwd unknown /proc/72983/task/72999/cwd (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72999 safe_time ceph rtd unknown /proc/72983/task/72999/root (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72999 safe_time ceph txt unknown /proc/72983/task/72999/exe (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 72999 safe_time ceph NOFD /proc/72983/task/72999/fd (opendir: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73000 safe_time ceph cwd unknown /proc/72983/task/73000/cwd (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73000 safe_time ceph rtd unknown /proc/72983/task/73000/root (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73000 safe_time ceph txt unknown /proc/72983/task/73000/exe (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73000 safe_time ceph NOFD /proc/72983/task/73000/fd (opendir: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73001 ceph-mds ceph cwd unknown /proc/72983/task/73001/cwd (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73001 ceph-mds ceph rtd unknown /proc/72983/task/73001/root (readlink: Permission denied) 2024-03-16T08:19:37.879 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73001 ceph-mds ceph txt unknown /proc/72983/task/73001/exe (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73001 ceph-mds ceph NOFD /proc/72983/task/73001/fd (opendir: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73131 safe_time ceph cwd unknown /proc/72983/task/73131/cwd (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73131 safe_time ceph rtd unknown /proc/72983/task/73131/root (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73131 safe_time ceph txt unknown /proc/72983/task/73131/exe (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 72983 73131 safe_time ceph NOFD /proc/72983/task/73131/fd (opendir: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 73343 root cwd unknown /proc/73343/cwd (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 73343 root rtd unknown /proc/73343/root (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 73343 root txt unknown /proc/73343/exe (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:conmon 73343 root NOFD /proc/73343/fd (opendir: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 73345 root cwd unknown /proc/73345/cwd (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 73345 root rtd unknown /proc/73345/root (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 73345 root txt unknown /proc/73345/exe (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:podman-in 73345 root NOFD /proc/73345/fd (opendir: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 ceph cwd unknown /proc/73347/cwd (readlink: Permission denied) 2024-03-16T08:19:37.880 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 ceph rtd unknown /proc/73347/root (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 ceph txt unknown /proc/73347/exe (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 ceph NOFD /proc/73347/fd (opendir: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73348 log ceph cwd unknown /proc/73347/task/73348/cwd (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73348 log ceph rtd unknown /proc/73347/task/73348/root (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73348 log ceph txt unknown /proc/73347/task/73348/exe (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73348 log ceph NOFD /proc/73347/task/73348/fd (opendir: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73350 msgr-work ceph cwd unknown /proc/73347/task/73350/cwd (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73350 msgr-work ceph rtd unknown /proc/73347/task/73350/root (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73350 msgr-work ceph txt unknown /proc/73347/task/73350/exe (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73350 msgr-work ceph NOFD /proc/73347/task/73350/fd (opendir: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73351 msgr-work ceph cwd unknown /proc/73347/task/73351/cwd (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73351 msgr-work ceph rtd unknown /proc/73347/task/73351/root (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73351 msgr-work ceph txt unknown /proc/73347/task/73351/exe (readlink: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73351 msgr-work ceph NOFD /proc/73347/task/73351/fd (opendir: Permission denied) 2024-03-16T08:19:37.881 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73352 msgr-work ceph cwd unknown /proc/73347/task/73352/cwd (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73352 msgr-work ceph rtd unknown /proc/73347/task/73352/root (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73352 msgr-work ceph txt unknown /proc/73347/task/73352/exe (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73352 msgr-work ceph NOFD /proc/73347/task/73352/fd (opendir: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73356 service ceph cwd unknown /proc/73347/task/73356/cwd (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73356 service ceph rtd unknown /proc/73347/task/73356/root (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73356 service ceph txt unknown /proc/73347/task/73356/exe (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73356 service ceph NOFD /proc/73347/task/73356/fd (opendir: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73357 admin_soc ceph cwd unknown /proc/73347/task/73357/cwd (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73357 admin_soc ceph rtd unknown /proc/73347/task/73357/root (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73357 admin_soc ceph txt unknown /proc/73347/task/73357/exe (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73357 admin_soc ceph NOFD /proc/73347/task/73357/fd (opendir: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73358 signal_ha ceph cwd unknown /proc/73347/task/73358/cwd (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73358 signal_ha ceph rtd unknown /proc/73347/task/73358/root (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73358 signal_ha ceph txt unknown /proc/73347/task/73358/exe (readlink: Permission denied) 2024-03-16T08:19:37.882 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73358 signal_ha ceph NOFD /proc/73347/task/73358/fd (opendir: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73359 io_contex ceph cwd unknown /proc/73347/task/73359/cwd (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73359 io_contex ceph rtd unknown /proc/73347/task/73359/root (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73359 io_contex ceph txt unknown /proc/73347/task/73359/exe (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73359 io_contex ceph NOFD /proc/73347/task/73359/fd (opendir: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73360 io_contex ceph cwd unknown /proc/73347/task/73360/cwd (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73360 io_contex ceph rtd unknown /proc/73347/task/73360/root (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73360 io_contex ceph txt unknown /proc/73347/task/73360/exe (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73360 io_contex ceph NOFD /proc/73347/task/73360/fd (opendir: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73361 ms_dispat ceph cwd unknown /proc/73347/task/73361/cwd (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73361 ms_dispat ceph rtd unknown /proc/73347/task/73361/root (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73361 ms_dispat ceph txt unknown /proc/73347/task/73361/exe (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73361 ms_dispat ceph NOFD /proc/73347/task/73361/fd (opendir: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73362 ms_local ceph cwd unknown /proc/73347/task/73362/cwd (readlink: Permission denied) 2024-03-16T08:19:37.883 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73362 ms_local ceph rtd unknown /proc/73347/task/73362/root (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73362 ms_local ceph txt unknown /proc/73347/task/73362/exe (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73362 ms_local ceph NOFD /proc/73347/task/73362/fd (opendir: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73363 safe_time ceph cwd unknown /proc/73347/task/73363/cwd (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73363 safe_time ceph rtd unknown /proc/73347/task/73363/root (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73363 safe_time ceph txt unknown /proc/73347/task/73363/exe (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73363 safe_time ceph NOFD /proc/73347/task/73363/fd (opendir: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73364 safe_time ceph cwd unknown /proc/73347/task/73364/cwd (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73364 safe_time ceph rtd unknown /proc/73347/task/73364/root (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73364 safe_time ceph txt unknown /proc/73347/task/73364/exe (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73364 safe_time ceph NOFD /proc/73347/task/73364/fd (opendir: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73365 ceph-mds ceph cwd unknown /proc/73347/task/73365/cwd (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73365 ceph-mds ceph rtd unknown /proc/73347/task/73365/root (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73365 ceph-mds ceph txt unknown /proc/73347/task/73365/exe (readlink: Permission denied) 2024-03-16T08:19:37.884 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73365 ceph-mds ceph NOFD /proc/73347/task/73365/fd (opendir: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73370 safe_time ceph cwd unknown /proc/73347/task/73370/cwd (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73370 safe_time ceph rtd unknown /proc/73347/task/73370/root (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73370 safe_time ceph txt unknown /proc/73347/task/73370/exe (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-mds 73347 73370 safe_time ceph NOFD /proc/73347/task/73370/fd (opendir: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 76733 root cwd unknown /proc/76733/cwd (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 76733 root rtd unknown /proc/76733/root (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 76733 root txt unknown /proc/76733/exe (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/4 76733 root NOFD /proc/76733/fd (opendir: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 79502 root cwd unknown /proc/79502/cwd (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 79502 root rtd unknown /proc/79502/root (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 79502 root txt unknown /proc/79502/exe (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/0 79502 root NOFD /proc/79502/fd (opendir: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-msgr 83401 root cwd unknown /proc/83401/cwd (readlink: Permission denied) 2024-03-16T08:19:37.885 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-msgr 83401 root rtd unknown /proc/83401/root (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-msgr 83401 root txt unknown /proc/83401/exe (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-msgr 83401 root NOFD /proc/83401/fd (opendir: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-watc 83402 root cwd unknown /proc/83402/cwd (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-watc 83402 root rtd unknown /proc/83402/root (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-watc 83402 root txt unknown /proc/83402/exe (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-watc 83402 root NOFD /proc/83402/fd (opendir: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-comp 83403 root cwd unknown /proc/83403/cwd (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-comp 83403 root rtd unknown /proc/83403/root (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-comp 83403 root txt unknown /proc/83403/exe (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:ceph-comp 83403 root NOFD /proc/83403/fd (opendir: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84758 root cwd unknown /proc/84758/cwd (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84758 root rtd unknown /proc/84758/root (readlink: Permission denied) 2024-03-16T08:19:37.886 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84758 root txt unknown /proc/84758/exe (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84758 root NOFD /proc/84758/fd (opendir: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84818 root cwd unknown /proc/84818/cwd (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84818 root rtd unknown /proc/84818/root (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84818 root txt unknown /proc/84818/exe (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sshd 84818 root NOFD /proc/84818/fd (opendir: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 84819 root cwd unknown /proc/84819/cwd (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 84819 root rtd unknown /proc/84819/root (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 84819 root txt unknown /proc/84819/exe (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:kworker/1 84819 root NOFD /proc/84819/fd (opendir: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 85411 root cwd unknown /proc/85411/cwd (readlink: Permission denied) 2024-03-16T08:19:37.887 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 85411 root rtd unknown /proc/85411/root (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 85411 root txt unknown /proc/85411/exe (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:sudo 85411 root NOFD /proc/85411/fd (opendir: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:daemon-he 85442 root cwd unknown /proc/85442/cwd (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:daemon-he 85442 root rtd unknown /proc/85442/root (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:daemon-he 85442 root txt unknown /proc/85442/exe (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:daemon-he 85442 root NOFD /proc/85442/fd (opendir: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:bash 85445 root cwd unknown /proc/85445/cwd (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:bash 85445 root rtd unknown /proc/85445/root (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:bash 85445 root txt unknown /proc/85445/exe (readlink: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:bash 85445 root NOFD /proc/85445/fd (opendir: Permission denied) 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T08:19:37.888 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu rtd DIR 8,1 4096 2 / 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu txt REG 8,1 40944 10098 /usr/bin/timeout 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-16T08:19:37.889 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu 0r FIFO 0,13 0t0 237609 pipe 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu 1w FIFO 0,13 0t0 237610 pipe 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:timeout 88366 ubuntu 2w FIFO 0,13 0t0 237611 pipe 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu rtd DIR 8,1 4096 2 / 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-16T08:19:37.890 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu 0r FIFO 0,13 0t0 237609 pipe 2024-03-16T08:19:37.891 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu 1w FIFO 0,13 0t0 237610 pipe 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu 2w FIFO 0,13 0t0 237611 pipe 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:fsync-tes 88384 ubuntu 255r REG 8,1 207 661291 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:sleep 89574 root cwd unknown /proc/89574/cwd (readlink: Permission denied) 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:sleep 89574 root rtd unknown /proc/89574/root (readlink: Permission denied) 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:sleep 89574 root txt unknown /proc/89574/exe (readlink: Permission denied) 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:sleep 89574 root NOFD /proc/89574/fd (opendir: Permission denied) 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu rtd DIR 8,1 4096 2 / 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 4487176 2250 /usr/lib64/libcrypto.so.3.0.7 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-16T08:19:37.892 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-16T08:19:37.893 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 0r FIFO 0,13 0t0 237609 pipe 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 1w FIFO 0,13 0t0 237610 pipe 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 2w FIFO 0,13 0t0 237611 pipe 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 3r DIR 0,20 0 1 /proc 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 4r DIR 0,20 0 235066 /proc/89576/fd 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 5w FIFO 0,13 0t0 235071 pipe 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89576 ubuntu 6r FIFO 0,13 0t0 235072 pipe 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu rtd DIR 8,1 4096 2 / 2024-03-16T08:19:37.894 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 4487176 2250 /usr/lib64/libcrypto.so.3.0.7 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-16T08:19:37.895 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu 4r FIFO 0,13 0t0 235071 pipe 2024-03-16T08:19:37.896 INFO:tasks.workunit.client.0.smithi049.stdout:lsof 89577 ubuntu 7w FIFO 0,13 0t0 235072 pipe 2024-03-16T08:19:37.896 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T08:19:37.897 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T08:19:37.951 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-03-16T08:19:37.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-16T08:19:38.248 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:37 smithi049 ceph-mon[31942]: pgmap v75: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-16T08:19:38.250 DEBUG:teuthology.parallel:result is None 2024-03-16T08:19:38.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T08:19:38.291 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T08:19:38.292 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-16T08:19:38.302 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:19:38.340 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-16T08:19:38.350 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-16T08:19:38.350 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T08:19:38.350 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:19:38.350 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-16T08:19:38.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:37 smithi129 ceph-mon[36561]: pgmap v75: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-16T08:19:38.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-16T08:19:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:37 smithi154 ceph-mon[36471]: pgmap v75: 97 pgs: 97 active+clean; 456 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-16T08:19:40.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:39 smithi049 ceph-mon[31942]: pgmap v76: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-03-16T08:19:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:39 smithi129 ceph-mon[36561]: pgmap v76: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-03-16T08:19:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:39 smithi154 ceph-mon[36471]: pgmap v76: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-03-16T08:19:42.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:41 smithi049 ceph-mon[31942]: pgmap v77: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:41 smithi129 ceph-mon[36561]: pgmap v77: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:41 smithi154 ceph-mon[36471]: pgmap v77: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:44.278 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:44 smithi154 ceph-mon[36471]: pgmap v78: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:44.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:44 smithi049 ceph-mon[31942]: pgmap v78: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:44 smithi129 ceph-mon[36561]: pgmap v78: 97 pgs: 97 active+clean; 515 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 15 op/s 2024-03-16T08:19:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:46 smithi049 ceph-mon[31942]: pgmap v79: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-03-16T08:19:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:46 smithi129 ceph-mon[36561]: pgmap v79: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-03-16T08:19:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:46 smithi154 ceph-mon[36471]: pgmap v79: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-03-16T08:19:46.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:19:46.670 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:19:46.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- 172.21.15.49:0/3873008838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc101780 msgr2=0x7f15cc101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 --2- 172.21.15.49:0/3873008838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc101780 0x7f15cc101c00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009930 tx=0x7f15c002bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:46.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- 172.21.15.49:0/3873008838 shutdown_connections 2024-03-16T08:19:46.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 --2- 172.21.15.49:0/3873008838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc102140 0x7f15cc10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 --2- 172.21.15.49:0/3873008838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc101780 0x7f15cc101c00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 --2- 172.21.15.49:0/3873008838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- 172.21.15.49:0/3873008838 >> 172.21.15.49:0/3873008838 conn(0x7f15cc0fd210 msgr2=0x7f15cc0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- 172.21.15.49:0/3873008838 shutdown_connections 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- 172.21.15.49:0/3873008838 wait complete. 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 Processor -- start 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.794+0000 7f15d268c640 1 -- start start 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 0x7f15cc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc111bf0 con 0x7f15cc102140 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc111a70 con 0x7f15cc107790 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d268c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc111d70 con 0x7f15cc101780 2024-03-16T08:19:46.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d0c29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d0c29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47264/0 (socket says 172.21.15.49:47264) 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58982/0 (socket says 172.21.15.49:58982) 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d0c29640 1 -- 172.21.15.49:0/409835249 learned_addr learned my addr 172.21.15.49:0/409835249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 0x7f15cc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 -- 172.21.15.49:0/409835249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 msgr2=0x7f15cc19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 -- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 msgr2=0x7f15cc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cbfff640 1 -- 172.21.15.49:0/409835249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc19d0a0 con 0x7f15cc101780 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15d0c29640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.795+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15cbfff640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 0x7f15cc1980b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15c97fa640 1 -- 172.21.15.49:0/409835249 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc009c30 con 0x7f15cc101780 2024-03-16T08:19:46.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc19d390 con 0x7f15cc101780 2024-03-16T08:19:46.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15c97fa640 1 -- 172.21.15.49:0/409835249 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc009dd0 con 0x7f15cc101780 2024-03-16T08:19:46.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15c97fa640 1 -- 172.21.15.49:0/409835249 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc010460 con 0x7f15cc101780 2024-03-16T08:19:46.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.796+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1a4c20 con 0x7f15cc101780 2024-03-16T08:19:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15c97fa640 1 -- 172.21.15.49:0/409835249 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f15bc003ef0 con 0x7f15cc101780 2024-03-16T08:19:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 Processor -- start 2024-03-16T08:19:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 -- start start 2024-03-16T08:19:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb65102260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb65066500 con 0x55fb6510fa60 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb6500ebe0 con 0x55fb65101e80 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.798+0000 7f15d527f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb650625a0 con 0x55fb651027a0 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15c97fa640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 0x7f15b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58986/0 (socket says 172.21.15.49:58986) 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 -- 172.21.15.49:0/1222474148 learned_addr learned my addr 172.21.15.49:0/1222474148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15d0c29640 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb65102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15d0c29640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 0x7f15b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15c97fa640 1 -- 172.21.15.49:0/409835249 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f15bc0ce870 con 0x7f15cc101780 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 -- 172.21.15.49:0/1222474148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 msgr2=0x55fb6510f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 -- 172.21.15.49:0/1222474148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 msgr2=0x55fb65102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb65102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 -- 172.21.15.49:0/1222474148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb650224b0 con 0x55fb6510fa60 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15d0c29640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 0x7f15b40a4e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f15c40049d0 tx=0x7f15c4009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15c002c240 tx=0x7f15c0005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3200d72662579b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.799+0000 7f15b27fc640 1 -- 172.21.15.49:0/1222474148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0045070 con 0x55fb6510fa60 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15b27fc640 1 -- 172.21.15.49:0/1222474148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0004a70 con 0x55fb6510fa60 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15b27fc640 1 -- 172.21.15.49:0/1222474148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0005140 con 0x55fb6510fa60 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 -- 172.21.15.49:0/1222474148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 msgr2=0x55fb65111e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15c002c240 tx=0x7f15c0005e60 comp rx=0 tx=0).stop 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 -- 172.21.15.49:0/1222474148 shutdown_connections 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb65111e50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 --2- 172.21.15.49:0/1222474148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb65102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 -- 172.21.15.49:0/1222474148 >> 172.21.15.49:0/1222474148 conn(0x55fb650fc880 msgr2=0x55fb650fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 -- 172.21.15.49:0/1222474148 shutdown_connections 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.800+0000 7f15d527f740 1 -- 172.21.15.49:0/1222474148 wait complete. 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 -- start start 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 Processor -- start 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fb651127c0 con 0x55fb6510fa60 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb65022160 con 0x55fb65101e80 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb65112940 con 0x55fb651027a0 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d527f740 10 client.0 _ll_register_callbacks cb 0x55fb65006150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d0c29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15d0c29640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58996/0 (socket says 172.21.15.49:58996) 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47278/0 (socket says 172.21.15.49:47278) 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15cbfff640 1 -- 172.21.15.49:0/1924498021 learned_addr learned my addr 172.21.15.49:0/1924498021 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.801+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 -- 172.21.15.49:0/1924498021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 msgr2=0x55fb6510cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 -- 172.21.15.49:0/1924498021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 msgr2=0x55fb6510d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 -- 172.21.15.49:0/1924498021 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb64e447d0 con 0x55fb65101e80 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15d0c29640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15bc097430 tx=0x7f15bc09ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ef4defb61299c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:19:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc020040 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c0004a70 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f15bc0cc290 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15bc0b8050 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15d527f740 10 client.26132 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.802+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fb64d7a3f0 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0004a70 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f15bc09a420 con 0x55fb65101e80 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 10 client.26132 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 20 client.26132 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6511fa00 0x55fb65121e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 4 client.26132 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb65073a40 con 0x55fb6511fa00 2024-03-16T08:19:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6511fa00 0x55fb65121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6511fa00 0x55fb65121e50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbf966cd75bf6c9 server_cookie=3b366ee5f87c8085 in_seq=0 out_seq=0 2024-03-16T08:19:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.803+0000 7f15ab7fe640 10 client.26132 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.804+0000 7f15ab7fe640 1 -- 172.21.15.49:0/1924498021 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fb65073a40 con 0x55fb6511fa00 2024-03-16T08:19:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.804+0000 7f15ab7fe640 10 client.26132 handle_command_reply: tid=0 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.804+0000 7f15d527f740 1 client.26132 shutdown 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 msgr2=0x55fb6510ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15bc097430 tx=0x7f15bc09ab90 comp rx=0 tx=0).stop 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 shutdown_connections 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6511fa00 0x55fb65121e50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb6510fa60 0x55fb6510d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6510cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 --2- 172.21.15.49:0/1924498021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510ca90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15ab7fe640 0 client.26132 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 >> 172.21.15.49:0/1924498021 conn(0x55fb650fc880 msgr2=0x55fb650fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 shutdown_connections 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 1 -- 172.21.15.49:0/1924498021 wait complete. 2024-03-16T08:19:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.805+0000 7f15d527f740 20 client.26132 trim_cache size 0 max 16384 2024-03-16T08:19:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.822+0000 7f15d527f740 1 Processor -- start 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.822+0000 7f15d527f740 1 -- start start 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6511b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ada50 con 0x55fb650fc880 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ab320 con 0x55fb65101e80 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d527f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1a9150 con 0x55fb651027a0 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59010/0 (socket says 172.21.15.49:59010) 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 -- 172.21.15.49:0/1476989632 learned_addr learned my addr 172.21.15.49:0/1476989632 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d0c29640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6511b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 -- 172.21.15.49:0/1476989632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 msgr2=0x55fb6511b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6511b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 -- 172.21.15.49:0/1476989632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 msgr2=0x55fb6510b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 -- 172.21.15.49:0/1476989632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb64d3a0f0 con 0x55fb650fc880 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15d0c29640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6511b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15cbfff640 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009610 tx=0x7f15bc0994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fb088871eea614 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15b1ffb640 1 -- 172.21.15.49:0/1476989632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc020070 con 0x55fb650fc880 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15b1ffb640 1 -- 172.21.15.49:0/1476989632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc097640 con 0x55fb650fc880 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.823+0000 7f15b1ffb640 1 -- 172.21.15.49:0/1476989632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0a3230 con 0x55fb650fc880 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- 172.21.15.49:0/1476989632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 msgr2=0x55fb6510f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009610 tx=0x7f15bc0994e0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- 172.21.15.49:0/1476989632 shutdown_connections 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb6511b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb65101e80 0x55fb6510b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 --2- 172.21.15.49:0/1476989632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb650fc880 0x55fb6510f1c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- 172.21.15.49:0/1476989632 >> 172.21.15.49:0/1476989632 conn(0x55fb65108460 msgr2=0x55fb650fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- 172.21.15.49:0/1476989632 shutdown_connections 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- 172.21.15.49:0/1476989632 wait complete. 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.824+0000 7f15d527f740 1 -- start start 2024-03-16T08:19:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 Processor -- start 2024-03-16T08:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb650fc880 0x55fb651134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb65113f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1a97e0 con 0x55fb65101e80 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1aac50 con 0x55fb650fc880 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1a9eb0 con 0x55fb651027a0 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d527f740 10 client.0 _ll_register_callbacks cb 0x7f15bc003850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59016/0 (socket says 172.21.15.49:59016) 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 -- 172.21.15.49:0/2290542228 learned_addr learned my addr 172.21.15.49:0/2290542228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cbfff640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb650fc880 0x55fb651134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15d0c29640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb65113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 -- 172.21.15.49:0/2290542228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 msgr2=0x55fb65113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb65113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 -- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb650fc880 msgr2=0x55fb651134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb650fc880 0x55fb651134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 -- 172.21.15.49:0/2290542228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6502d9d0 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15cb7fe640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15c002c170 tx=0x7f15c0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccfa96d77bb578a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.825+0000 7f15ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0045040 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f15c0050050 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15bc097640 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15c002c650 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 10 client.16257 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fb65185ee0 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc097640 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f15c004a450 con 0x55fb65101e80 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 10 client.16257 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 20 client.16257 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6510fa60 0x55fb651882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 4 client.16257 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb6506dd90 con 0x55fb6510fa60 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.826+0000 7f15cbfff640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6510fa60 0x55fb651882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15ab7fe640 10 client.16257 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15cbfff640 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6510fa60 0x55fb651882d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6191ce035092c231 server_cookie=178600c3df20de70 in_seq=0 out_seq=0 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15ab7fe640 1 -- 172.21.15.49:0/2290542228 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb6506dd90 con 0x55fb6510fa60 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15ab7fe640 10 client.16257 handle_command_reply: tid=0 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 client.16257 shutdown 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 msgr2=0x55fb651139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15c002c170 tx=0x7f15c0004600 comp rx=0 tx=0).stop 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 shutdown_connections 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fb6510fa60 0x55fb651882d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb651027a0 0x55fb65113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fb65101e80 0x55fb651139f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15ab7fe640 0 client.16257 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 --2- 172.21.15.49:0/2290542228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb650fc880 0x55fb651134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 >> 172.21.15.49:0/2290542228 conn(0x55fb65108460 msgr2=0x55fb6510ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 shutdown_connections 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 1 -- 172.21.15.49:0/2290542228 wait complete. 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.827+0000 7f15d527f740 20 client.16257 trim_cache size 0 max 16384 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:19:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:46.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 msgr2=0x7f15b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 0x7f15b40a4e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f15c40049d0 tx=0x7f15c4009500 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 msgr2=0x7f15cc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 0x7f15cc1980b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc00dde0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 shutdown_connections 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f15b40a2970 0x7f15b40a4e30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15cc107790 0x7f15cc19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc102140 0x7f15cc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 --2- 172.21.15.49:0/409835249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15cc101780 0x7f15cc1980b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 >> 172.21.15.49:0/409835249 conn(0x7f15cc0fd210 msgr2=0x7f15cc0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 shutdown_connections 2024-03-16T08:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.828+0000 7f15d268c640 1 -- 172.21.15.49:0/409835249 wait complete. 2024-03-16T08:19:46.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:19:46.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:19:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- 172.21.15.49:0/891692546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c1017a0 msgr2=0x7fc67c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 --2- 172.21.15.49:0/891692546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c1017a0 0x7fc67c101c20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009930 tx=0x7fc66c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- 172.21.15.49:0/891692546 shutdown_connections 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 --2- 172.21.15.49:0/891692546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c102160 0x7fc67c10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 --2- 172.21.15.49:0/891692546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c1017a0 0x7fc67c101c20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 --2- 172.21.15.49:0/891692546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 0x7fc67c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- 172.21.15.49:0/891692546 >> 172.21.15.49:0/891692546 conn(0x7fc67c0fd250 msgr2=0x7fc67c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- 172.21.15.49:0/891692546 shutdown_connections 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- 172.21.15.49:0/891692546 wait complete. 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 Processor -- start 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.948+0000 7fc6829e6640 1 -- start start 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c1017a0 0x7fc67c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 0x7fc67c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111b80 con 0x7fc67c102160 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111a00 con 0x7fc67c1017a0 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6829e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111d00 con 0x7fc67c1077b0 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59032/0 (socket says 172.21.15.49:59032) 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 -- 172.21.15.49:0/1543593193 learned_addr learned my addr 172.21.15.49:0/1543593193 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67bfff640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c1017a0 0x7fc67c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc680f83640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 0x7fc67c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 -- 172.21.15.49:0/1543593193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 msgr2=0x7fc67c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 0x7fc67c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 -- 172.21.15.49:0/1543593193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c1017a0 msgr2=0x7fc67c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c1017a0 0x7fc67c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 -- 172.21.15.49:0/1543593193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c19d0d0 con 0x7fc67c102160 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc67b7fe640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009230 tx=0x7fc66c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.949+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c041070 con 0x7fc67c102160 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.950+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c004450 con 0x7fc67c102160 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.950+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c19d360 con 0x7fc67c102160 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.950+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c003e50 con 0x7fc67c102160 2024-03-16T08:19:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.950+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c1a4c50 con 0x7fc67c102160 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.951+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c068a20 con 0x7fc67c102160 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.952+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc66c044070 con 0x7fc67c102160 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.952+0000 7fc6797fa640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 0x7fc6680a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.952+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc66c0f09d0 con 0x7fc67c102160 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.952+0000 7fc67bfff640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 0x7fc6680a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.953+0000 7fc67bfff640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 0x7fc6680a4fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6640095f0 tx=0x7fc664005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:46.956+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc66c0bd240 con 0x7fc67c102160 2024-03-16T08:19:47.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.180+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67c199430 con 0x7fc67c102160 2024-03-16T08:19:47.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.181+0000 7fc6797fa640 1 -- 172.21.15.49:0/1543593193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fc66c0c12e0 con 0x7fc67c102160 2024-03-16T08:19:47.181 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:19:47.181 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:19:47.181 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 msgr2=0x7fc6680a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 0x7fc6680a4fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6640095f0 tx=0x7fc664005f90 comp rx=0 tx=0).stop 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 msgr2=0x7fc67c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009230 tx=0x7fc66c02cb30 comp rx=0 tx=0).stop 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 shutdown_connections 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc6680a2b10 0x7fc6680a4fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc67c1077b0 0x7fc67c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc67c102160 0x7fc67c198620 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 --2- 172.21.15.49:0/1543593193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc67c1017a0 0x7fc67c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 >> 172.21.15.49:0/1543593193 conn(0x7fc67c0fd250 msgr2=0x7fc67c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 shutdown_connections 2024-03-16T08:19:47.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.183+0000 7fc6829e6640 1 -- 172.21.15.49:0/1543593193 wait complete. 2024-03-16T08:19:47.195 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:19:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- 172.21.15.49:0/2911591547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4100650 msgr2=0x7f30a4100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 --2- 172.21.15.49:0/2911591547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4100650 0x7f30a4100ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3098009890 tx=0x7f309802bb70 comp rx=0 tx=0).stop 2024-03-16T08:19:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- 172.21.15.49:0/2911591547 shutdown_connections 2024-03-16T08:19:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 --2- 172.21.15.49:0/2911591547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4101010 0x7f30a410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 --2- 172.21.15.49:0/2911591547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4100650 0x7f30a4100ad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 --2- 172.21.15.49:0/2911591547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4106660 0x7f30a4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- 172.21.15.49:0/2911591547 >> 172.21.15.49:0/2911591547 conn(0x7f30a40777e0 msgr2=0x7f30a4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- 172.21.15.49:0/2911591547 shutdown_connections 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- 172.21.15.49:0/2911591547 wait complete. 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 Processor -- start 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.305+0000 7f30aa13b640 1 -- start start 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4100650 0x7f30a41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4106660 0x7f30a419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4110a70 con 0x7f30a4101010 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41108f0 con 0x7f30a4100650 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30aa13b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4110bf0 con 0x7f30a4106660 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59052/0 (socket says 172.21.15.49:59052) 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4100650 0x7f30a41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 -- 172.21.15.49:0/4185121238 learned_addr learned my addr 172.21.15.49:0/4185121238 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a3fff640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4106660 0x7f30a419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 -- 172.21.15.49:0/4185121238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4106660 msgr2=0x7f30a419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4106660 0x7f30a419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 -- 172.21.15.49:0/4185121238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4100650 msgr2=0x7f30a41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4100650 0x7f30a41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 -- 172.21.15.49:0/4185121238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a419d0b0 con 0x7f30a4101010 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f30980057c0 tx=0x7f30980045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.306+0000 7f30a0ff9640 1 -- 172.21.15.49:0/4185121238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098041020 con 0x7f30a4101010 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.307+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a419d340 con 0x7f30a4101010 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.307+0000 7f30a0ff9640 1 -- 172.21.15.49:0/4185121238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309802c9f0 con 0x7f30a4101010 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.307+0000 7f30a0ff9640 1 -- 172.21.15.49:0/4185121238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098039040 con 0x7f30a4101010 2024-03-16T08:19:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.307+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a4c30 con 0x7f30a4101010 2024-03-16T08:19:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30a0ff9640 1 -- 172.21.15.49:0/4185121238 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3098044020 con 0x7f30a4101010 2024-03-16T08:19:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 Processor -- start 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 -- start start 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 0x55cfb8fb07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8f0b490 con 0x55cfb8fa7830 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8eb3c90 con 0x55cfb8fb0d30 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.309+0000 7f30ace08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8f07530 con 0x55cfb8fa8150 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59054/0 (socket says 172.21.15.49:59054) 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 -- 172.21.15.49:0/4252899414 learned_addr learned my addr 172.21.15.49:0/4252899414 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a0ff9640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 0x7f308c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 0x55cfb8fb07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 0x7f308c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a0ff9640 1 -- 172.21.15.49:0/4185121238 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f30980f0630 con 0x7f30a4101010 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 -- 172.21.15.49:0/4252899414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fb0d30 msgr2=0x55cfb8fb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 -- 172.21.15.49:0/4252899414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 msgr2=0x55cfb8fa7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 -- 172.21.15.49:0/4252899414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8f6e500 con 0x55cfb8fa8150 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a37fe640 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 0x55cfb8fb07f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3094009c30 tx=0x7f3094011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fc30d05be3e9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f30a3fff640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 0x7f308c0a4e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f309c009fd0 tx=0x7f309c004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f3089ffb640 1 -- 172.21.15.49:0/4252899414 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309401e070 con 0x55cfb8fa8150 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f3089ffb640 1 -- 172.21.15.49:0/4252899414 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30940035f0 con 0x55cfb8fa8150 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.310+0000 7f3089ffb640 1 -- 172.21.15.49:0/4252899414 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094019690 con 0x55cfb8fa8150 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- 172.21.15.49:0/4252899414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 msgr2=0x55cfb8fb07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 0x55cfb8fb07f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3094009c30 tx=0x7f3094011810 comp rx=0 tx=0).stop 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- 172.21.15.49:0/4252899414 shutdown_connections 2024-03-16T08:19:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fa8150 0x55cfb8fb07f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 --2- 172.21.15.49:0/4252899414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fa7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- 172.21.15.49:0/4252899414 >> 172.21.15.49:0/4252899414 conn(0x55cfb8fa1f60 msgr2=0x55cfb8fa2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- 172.21.15.49:0/4252899414 shutdown_connections 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- 172.21.15.49:0/4252899414 wait complete. 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.311+0000 7f30ace08740 1 -- start start 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 Processor -- start 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 0x55cfb8fae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8fb3b20 con 0x55cfb8fa7830 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8ec7230 con 0x55cfb8fa8150 2024-03-16T08:19:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8fb3ca0 con 0x55cfb8fb0d30 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30ace08740 10 client.0 _ll_register_callbacks cb 0x55cfb8f03010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 0x55cfb8fae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59056/0 (socket says 172.21.15.49:59056) 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:59932/0 (socket says 172.21.15.49:59932) 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 -- 172.21.15.49:0/1908796963 learned_addr learned my addr 172.21.15.49:0/1908796963 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 -- 172.21.15.49:0/1908796963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 msgr2=0x55cfb8fae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 -- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 msgr2=0x55cfb8fae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 0x55cfb8fae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a37fe640 1 -- 172.21.15.49:0/1908796963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb8ed6100 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a3fff640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.312+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 0x55cfb8fae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30a37fe640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6368b449541334 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094028020 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3094005470 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30940035f0 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3094033050 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 10 client.16275 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cfb8c1f3f0 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30940035f0 con 0x55cfb8fa7830 2024-03-16T08:19:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3094021030 con 0x55cfb8fa7830 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 10 client.16275 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 10 client.16275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 20 client.16275 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb8fc10e0 0x55cfb8fc34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 4 client.16275 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.313+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb8f189d0 con 0x55cfb8fc10e0 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.314+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb8fc10e0 0x55cfb8fc34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.314+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb8fc10e0 0x55cfb8fc34d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=372b6eacac3140e6 server_cookie=e07a71590b69071f in_seq=0 out_seq=0 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.314+0000 7f3082ffd640 10 client.16275 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.314+0000 7f3082ffd640 1 -- 172.21.15.49:0/1908796963 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cfb8f189d0 con 0x55cfb8fc10e0 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.314+0000 7f3082ffd640 10 client.16275 handle_command_reply: tid=0 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 client.16275 shutdown 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 msgr2=0x55cfb8fadde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094003bd0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 shutdown_connections 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb8fc10e0 0x55cfb8fc34d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa8150 0x55cfb8fae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f3082ffd640 0 client.16275 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 --2- 172.21.15.49:0/1908796963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa7830 0x55cfb8fadde0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 >> 172.21.15.49:0/1908796963 conn(0x55cfb8fa1f60 msgr2=0x55cfb8fa3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 shutdown_connections 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 1 -- 172.21.15.49:0/1908796963 wait complete. 2024-03-16T08:19:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.315+0000 7f30ace08740 20 client.16275 trim_cache size 0 max 16384 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 Processor -- start 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 -- start start 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fc8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41afd50 con 0x55cfb8fa8150 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ad620 con 0x55cfb8fa7830 2024-03-16T08:19:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30ace08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ab450 con 0x55cfb8fb0d30 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59062/0 (socket says 172.21.15.49:59062) 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30a2ffd640 1 -- 172.21.15.49:0/1548991286 learned_addr learned my addr 172.21.15.49:0/1548991286 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30a37fe640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.328+0000 7f30a3fff640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fc8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 -- 172.21.15.49:0/1548991286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 msgr2=0x55cfb8fc8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fc8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 -- 172.21.15.49:0/1548991286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 msgr2=0x55cfb8fb0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 -- 172.21.15.49:0/1548991286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8fc3840 con 0x55cfb8fa8150 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a37fe640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f30980bcd80 tx=0x7f309802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15d4fea8507d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30897fa640 1 -- 172.21.15.49:0/1548991286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980bcdf0 con 0x55cfb8fa8150 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30897fa640 1 -- 172.21.15.49:0/1548991286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980c0f40 con 0x55cfb8fa8150 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30897fa640 1 -- 172.21.15.49:0/1548991286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098032c70 con 0x55cfb8fa8150 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30ace08740 1 -- 172.21.15.49:0/1548991286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 msgr2=0x55cfb8fbc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.329+0000 7f30ace08740 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f30980bcd80 tx=0x7f309802bb70 comp rx=0 tx=0).stop 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- 172.21.15.49:0/1548991286 shutdown_connections 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8fc8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8fbc8c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- 172.21.15.49:0/1548991286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- 172.21.15.49:0/1548991286 >> 172.21.15.49:0/1548991286 conn(0x55cfb8fa1f60 msgr2=0x55cfb8fa9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- 172.21.15.49:0/1548991286 shutdown_connections 2024-03-16T08:19:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- 172.21.15.49:0/1548991286 wait complete. 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- start start 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 Processor -- start 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8faa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41abae0 con 0x55cfb8fa8150 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41acf50 con 0x55cfb8fa7830 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ac1b0 con 0x55cfb8fb0d30 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30ace08740 10 client.0 _ll_register_callbacks cb 0x7f309401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8faa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59064/0 (socket says 172.21.15.49:59064) 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 -- 172.21.15.49:0/957974435 learned_addr learned my addr 172.21.15.49:0/957974435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.330+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47368/0 (socket says 172.21.15.49:47368) 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 -- 172.21.15.49:0/957974435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 msgr2=0x55cfb8faa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8faa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 -- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 msgr2=0x55cfb8fb9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 -- 172.21.15.49:0/957974435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb8ce97d0 con 0x55cfb8fa8150 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a37fe640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30a2ffd640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f309802c180 tx=0x7f30980326d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c756d6760edbbd12 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980bcdf0 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3098034760 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30980c0f40 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30980b9460 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30ace08740 10 client.16281 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cfb90262e0 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980c0f40 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.331+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f309803c030 con 0x55cfb8fa8150 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 10 client.16281 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 20 client.16281 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb90264e0 0x55cfb90288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 4 client.16281 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfb8f12d20 con 0x55cfb90264e0 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30a37fe640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb90264e0 0x55cfb90288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f3082ffd640 10 client.16281 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30a37fe640 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb90264e0 0x55cfb90288d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dec81ad54b967739 server_cookie=b0210a4c51d4e178 in_seq=0 out_seq=0 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f3082ffd640 1 -- 172.21.15.49:0/957974435 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfb8f12d20 con 0x55cfb90264e0 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f3082ffd640 10 client.16281 handle_command_reply: tid=0 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 client.16281 shutdown 2024-03-16T08:19:47.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 msgr2=0x55cfb8faa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f309802c180 tx=0x7f30980326d0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 shutdown_connections 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cfb90264e0 0x55cfb90288d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfb8fb0d30 0x55cfb8faa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f3082ffd640 0 client.16281 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfb8fa8150 0x55cfb8faa050 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 --2- 172.21.15.49:0/957974435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfb8fa7830 0x55cfb8fb9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.332+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 >> 172.21.15.49:0/957974435 conn(0x55cfb8fa1f60 msgr2=0x55cfb8fa0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 shutdown_connections 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30ace08740 1 -- 172.21.15.49:0/957974435 wait complete. 2024-03-16T08:19:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30ace08740 20 client.16281 trim_cache size 0 max 16384 2024-03-16T08:19:47.342 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 msgr2=0x7f308c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 0x7f308c0a4e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f309c009fd0 tx=0x7f309c004ab0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 msgr2=0x7f30a4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f30980057c0 tx=0x7f30980045d0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 shutdown_connections 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f308c0a2970 0x7f308c0a4e30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4106660 0x7f30a419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30a4101010 0x7f30a4198600 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 --2- 172.21.15.49:0/4185121238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a4100650 0x7f30a41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 >> 172.21.15.49:0/4185121238 conn(0x7f30a40777e0 msgr2=0x7f30a4068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 shutdown_connections 2024-03-16T08:19:47.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.333+0000 7f30aa13b640 1 -- 172.21.15.49:0/4185121238 wait complete. 2024-03-16T08:19:47.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 -- 172.21.15.49:0/166820232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610106660 msgr2=0x7f8610106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 --2- 172.21.15.49:0/166820232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610106660 0x7f8610106a40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8604009930 tx=0x7f860402bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 -- 172.21.15.49:0/166820232 shutdown_connections 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 --2- 172.21.15.49:0/166820232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610101010 0x7f861010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 --2- 172.21.15.49:0/166820232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610100650 0x7f8610100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 --2- 172.21.15.49:0/166820232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610106660 0x7f8610106a40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 -- 172.21.15.49:0/166820232 >> 172.21.15.49:0/166820232 conn(0x7f86100777e0 msgr2=0x7f8610077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 -- 172.21.15.49:0/166820232 shutdown_connections 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.456+0000 7f8617137640 1 -- 172.21.15.49:0/166820232 wait complete. 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 Processor -- start 2024-03-16T08:19:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 -- start start 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610100650 0x7f8610198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610106660 0x7f861019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8610110a30 con 0x7f8610101010 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86101108b0 con 0x7f8610100650 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8617137640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8610110bb0 con 0x7f8610106660 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59086/0 (socket says 172.21.15.49:59086) 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f860ffff640 1 -- 172.21.15.49:0/1273365398 learned_addr learned my addr 172.21.15.49:0/1273365398 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f8614ed3640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610100650 0x7f8610198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.457+0000 7f86156d4640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610106660 0x7f861019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 -- 172.21.15.49:0/1273365398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610106660 msgr2=0x7f861019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610106660 0x7f861019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 -- 172.21.15.49:0/1273365398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610100650 msgr2=0x7f8610198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610100650 0x7f8610198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 -- 172.21.15.49:0/1273365398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861019d050 con 0x7f8610101010 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860ffff640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e9f0 tx=0x7f85fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860dffb640 1 -- 172.21.15.49:0/1273365398 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00ccc0 con 0x7f8610101010 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861019d340 con 0x7f8610101010 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860dffb640 1 -- 172.21.15.49:0/1273365398 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc010910 con 0x7f8610101010 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a4bd0 con 0x7f8610101010 2024-03-16T08:19:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.458+0000 7f860dffb640 1 -- 172.21.15.49:0/1273365398 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc018840 con 0x7f8610101010 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.460+0000 7f860dffb640 1 -- 172.21.15.49:0/1273365398 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f85fc010ac0 con 0x7f8610101010 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 Processor -- start 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 -- start start 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 0x55b6c31e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3149490 con 0x55b6c31e5560 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6c30f1c90 con 0x55b6c31f3140 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8619df8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3145530 con 0x55b6c31e5e80 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f86156d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 0x55b6c31e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860dffb640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 0x7f86000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47396/0 (socket says 172.21.15.49:47396) 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f86156d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 0x55b6c31e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59100/0 (socket says 172.21.15.49:59100) 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860ffff640 1 -- 172.21.15.49:0/2429877006 learned_addr learned my addr 172.21.15.49:0/2429877006 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f86156d4640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 0x7f86000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860dffb640 1 -- 172.21.15.49:0/1273365398 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f85fc0cf1e0 con 0x7f8610101010 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 -- 172.21.15.49:0/2429877006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 msgr2=0x55b6c31f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 -- 172.21.15.49:0/2429877006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 msgr2=0x55b6c31e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 0x55b6c31e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f8614ed3640 1 -- 172.21.15.49:0/2429877006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c31ac500 con 0x55b6c31e5e80 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.461+0000 7f860ffff640 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8614ed3640 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f2c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8604009230 tx=0x7f8604009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8b64dd72e8e289 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f86156d4640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 0x7f86000a4e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86080094d0 tx=0x7f8608004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f85faffd640 1 -- 172.21.15.49:0/2429877006 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604042070 con 0x55b6c31e5e80 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f85faffd640 1 -- 172.21.15.49:0/2429877006 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860402c690 con 0x55b6c31e5e80 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f85faffd640 1 -- 172.21.15.49:0/2429877006 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604004b00 con 0x55b6c31e5e80 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 -- 172.21.15.49:0/2429877006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 msgr2=0x55b6c31f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f2c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8604009230 tx=0x7f8604009610 comp rx=0 tx=0).stop 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 -- 172.21.15.49:0/2429877006 shutdown_connections 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f2c00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 --2- 172.21.15.49:0/2429877006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5560 0x55b6c31e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.462+0000 7f8619df8740 1 -- 172.21.15.49:0/2429877006 >> 172.21.15.49:0/2429877006 conn(0x55b6c31dff60 msgr2=0x55b6c31e0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- 172.21.15.49:0/2429877006 shutdown_connections 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- 172.21.15.49:0/2429877006 wait complete. 2024-03-16T08:19:47.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- start start 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 Processor -- start 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31e5560 0x55b6c31f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b6c31f5ea0 con 0x55b6c31f3140 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3105230 con 0x55b6c31e5560 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b6c31f6020 con 0x55b6c31e5e80 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8619df8740 10 client.0 _ll_register_callbacks cb 0x55b6c3141010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f86156d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f8614ed3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31e5560 0x55b6c31f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f860ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f86156d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59102/0 (socket says 172.21.15.49:59102) 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.463+0000 7f86156d4640 1 -- 172.21.15.49:0/3653458407 learned_addr learned my addr 172.21.15.49:0/3653458407 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 -- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 msgr2=0x55b6c31f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 -- 172.21.15.49:0/3653458407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31e5560 msgr2=0x55b6c31f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31e5560 0x55b6c31f01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 -- 172.21.15.49:0/3653458407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c3114100 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f86156d4640 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f860800fbb0 tx=0x7f8608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf19a5f3fd953b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f85effff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608022450 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6c31569d0 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f86080225f0 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f86080227f0 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f8619df8740 10 client.16293 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b6c2e5d3f0 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.464+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b6c31569d0 con 0x55b6c31f3140 2024-03-16T08:19:47.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f860801f040 con 0x55b6c31f3140 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 10 client.16293 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 10 client.16293 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 20 client.16293 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c3203490 0x55b6c32058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 4 client.16293 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c3150d20 con 0x55b6c3203490 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8614ed3640 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c3203490 0x55b6c32058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f8614ed3640 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c3203490 0x55b6c32058d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96d9fbdd26e78ccf server_cookie=c78c590b0025bb5b in_seq=0 out_seq=0 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.465+0000 7f85effff640 10 client.16293 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f85effff640 1 -- 172.21.15.49:0/3653458407 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b6c3150d20 con 0x55b6c3203490 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f85effff640 10 client.16293 handle_command_reply: tid=0 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 client.16293 shutdown 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 msgr2=0x55b6c31f0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f860800fbb0 tx=0x7f8608004a40 comp rx=0 tx=0).stop 2024-03-16T08:19:47.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 shutdown_connections 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c3203490 0x55b6c32058d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31f3140 0x55b6c31f0c20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5e80 0x55b6c31f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 --2- 172.21.15.49:0/3653458407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31e5560 0x55b6c31f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f85effff640 0 client.16293 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 >> 172.21.15.49:0/3653458407 conn(0x55b6c31dff60 msgr2=0x55b6c31e1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 shutdown_connections 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.466+0000 7f8619df8740 1 -- 172.21.15.49:0/3653458407 wait complete. 2024-03-16T08:19:47.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.467+0000 7f8619df8740 20 client.16293 trim_cache size 0 max 16384 2024-03-16T08:19:47.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 Processor -- start 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 -- start start 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c320b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ada40 con 0x55b6c31e5e80 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ab310 con 0x55b6c31f3140 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8619df8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a9140 con 0x55b6c31e5560 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59108/0 (socket says 172.21.15.49:59108) 2024-03-16T08:19:47.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 -- 172.21.15.49:0/3656188119 learned_addr learned my addr 172.21.15.49:0/3656188119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f86156d4640 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c320b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f8614ed3640 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 -- 172.21.15.49:0/3656188119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 msgr2=0x55b6c31fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 -- 172.21.15.49:0/3656188119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 msgr2=0x55b6c320b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c320b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 -- 172.21.15.49:0/3656188119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c31e5cb0 con 0x55b6c31e5e80 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.480+0000 7f860ffff640 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f85fc097cf0 tx=0x7f85fc086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc4150065717e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f85fa7fc640 1 -- 172.21.15.49:0/3656188119 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc09bf90 con 0x55b6c31e5e80 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f85fa7fc640 1 -- 172.21.15.49:0/3656188119 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc0cf030 con 0x55b6c31e5e80 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f85fa7fc640 1 -- 172.21.15.49:0/3656188119 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0ccc00 con 0x55b6c31e5e80 2024-03-16T08:19:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- 172.21.15.49:0/3656188119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 msgr2=0x55b6c31dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f85fc097cf0 tx=0x7f85fc086040 comp rx=0 tx=0).stop 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- 172.21.15.49:0/3656188119 shutdown_connections 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c320b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31dee60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 --2- 172.21.15.49:0/3656188119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- 172.21.15.49:0/3656188119 >> 172.21.15.49:0/3656188119 conn(0x55b6c31dff60 msgr2=0x55b6c31de750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- 172.21.15.49:0/3656188119 shutdown_connections 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- 172.21.15.49:0/3656188119 wait complete. 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 -- start start 2024-03-16T08:19:47.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.481+0000 7f8619df8740 1 Processor -- start 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a97d0 con 0x55b6c31e5e80 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86101aac40 con 0x55b6c31f3140 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a9ea0 con 0x55b6c31e5560 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8619df8740 10 client.0 _ll_register_callbacks cb 0x7f8608013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f8614ed3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f86156d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59124/0 (socket says 172.21.15.49:59124) 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 -- 172.21.15.49:0/3494636337 learned_addr learned my addr 172.21.15.49:0/3494636337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 -- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 msgr2=0x55b6c31ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 -- 172.21.15.49:0/3494636337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 msgr2=0x55b6c31ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 -- 172.21.15.49:0/3494636337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c2f277d0 con 0x55b6c31e5e80 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f860ffff640 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f85fc098260 tx=0x7f85fc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf989f127b88099 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.482+0000 7f85effff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0cce00 con 0x55b6c31e5e80 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f85fc09cbc0 con 0x55b6c31e5e80 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85fc0cf030 con 0x55b6c31e5e80 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85fc09b890 con 0x55b6c31e5e80 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 10 client.16299 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b6c3269940 con 0x55b6c31e5e80 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc0cf030 con 0x55b6c31e5e80 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f85fc0a6050 con 0x55b6c31e5e80 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 10 client.16299 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 20 client.16299 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c326a0f0 0x55b6c326c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 4 client.16299 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6c30ce6a0 con 0x55b6c326a0f0 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.483+0000 7f8614ed3640 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c326a0f0 0x55b6c326c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f85effff640 10 client.16299 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8614ed3640 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c326a0f0 0x55b6c326c510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f2b3950c0d37673 server_cookie=1f4fbb3fd9de4245 in_seq=0 out_seq=0 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f85effff640 1 -- 172.21.15.49:0/3494636337 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6c30ce6a0 con 0x55b6c326a0f0 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f85effff640 10 client.16299 handle_command_reply: tid=0 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 client.16299 shutdown 2024-03-16T08:19:47.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 msgr2=0x55b6c31ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f85fc098260 tx=0x7f85fc00ce60 comp rx=0 tx=0).stop 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 shutdown_connections 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b6c326a0f0 0x55b6c326c510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6c31f3140 0x55b6c31ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6c31e5e80 0x55b6c31ed2a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f85effff640 0 client.16299 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 --2- 172.21.15.49:0/3494636337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6c31e5560 0x55b6c31ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 >> 172.21.15.49:0/3494636337 conn(0x55b6c31dff60 msgr2=0x55b6c31defe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 shutdown_connections 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 1 -- 172.21.15.49:0/3494636337 wait complete. 2024-03-16T08:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.484+0000 7f8619df8740 20 client.16299 trim_cache size 0 max 16384 2024-03-16T08:19:47.492 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 msgr2=0x7f86000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 0x7f86000a4e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86080094d0 tx=0x7f8608004ab0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 msgr2=0x7f86101985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e9f0 tx=0x7f85fc00edf0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 shutdown_connections 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f86000a2970 0x7f86000a4e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8610106660 0x7f861019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8610101010 0x7f86101985a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 --2- 172.21.15.49:0/1273365398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8610100650 0x7f8610198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 >> 172.21.15.49:0/1273365398 conn(0x7f86100777e0 msgr2=0x7f8610068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 shutdown_connections 2024-03-16T08:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.485+0000 7f8617137640 1 -- 172.21.15.49:0/1273365398 wait complete. 2024-03-16T08:19:47.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:19:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- 172.21.15.49:0/3055090104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97601010c0 msgr2=0x7f976010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 --2- 172.21.15.49:0/3055090104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97601010c0 0x7f976010d5d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9750009cd0 tx=0x7f975002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- 172.21.15.49:0/3055090104 shutdown_connections 2024-03-16T08:19:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 --2- 172.21.15.49:0/3055090104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97601010c0 0x7f976010d5d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 --2- 172.21.15.49:0/3055090104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760100700 0x7f9760100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 --2- 172.21.15.49:0/3055090104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 0x7f9760106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- 172.21.15.49:0/3055090104 >> 172.21.15.49:0/3055090104 conn(0x7f9760077840 msgr2=0x7f9760077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- 172.21.15.49:0/3055090104 shutdown_connections 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- 172.21.15.49:0/3055090104 wait complete. 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 Processor -- start 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.607+0000 7f976817e640 1 -- start start 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97601010c0 0x7f97600fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 0x7f97600fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9760110b20 con 0x7f9760100700 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97601109a0 con 0x7f97601010c0 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9760110ca0 con 0x7f9760106710 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59150/0 (socket says 172.21.15.49:59150) 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 -- 172.21.15.49:0/168664047 learned_addr learned my addr 172.21.15.49:0/168664047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765719640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97601010c0 0x7f97600fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976671b640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 0x7f97600fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 -- 172.21.15.49:0/168664047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 msgr2=0x7f97600fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 0x7f97600fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 -- 172.21.15.49:0/168664047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97601010c0 msgr2=0x7f97600fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97601010c0 0x7f97600fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 -- 172.21.15.49:0/168664047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a66a0 con 0x7f9760100700 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f9765f1a640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f975800e9f0 tx=0x7f975800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f975effd640 1 -- 172.21.15.49:0/168664047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800cd80 con 0x7f9760100700 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f975effd640 1 -- 172.21.15.49:0/168664047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758005500 con 0x7f9760100700 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a6990 con 0x7f9760100700 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f975effd640 1 -- 172.21.15.49:0/168664047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758010690 con 0x7f9760100700 2024-03-16T08:19:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.608+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a6ee0 con 0x7f9760100700 2024-03-16T08:19:47.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f975effd640 1 -- 172.21.15.49:0/168664047 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f97580046c0 con 0x7f9760100700 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 Processor -- start 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 -- start start 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 0x55b6459ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b645906490 con 0x55b6459abd30 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6458aec90 con 0x55b6459a3150 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.611+0000 7f976ad6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b645902530 con 0x55b6459a2830 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 0x55b6459ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f975effd640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 0x7f97480a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f976671b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59154/0 (socket says 172.21.15.49:59154) 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 -- 172.21.15.49:0/2116647660 learned_addr learned my addr 172.21.15.49:0/2116647660 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f976671b640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 0x7f97480a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f975effd640 1 -- 172.21.15.49:0/168664047 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f97580cf090 con 0x7f9760100700 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 -- 172.21.15.49:0/2116647660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 msgr2=0x55b6459a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 -- 172.21.15.49:0/2116647660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 msgr2=0x55b6459ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 0x55b6459ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 -- 172.21.15.49:0/2116647660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b645969500 con 0x55b6459abd30 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765f1a640 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 0x55b6459ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9765719640 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9754009c90 tx=0x7f9754011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb60a931408f8a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9747fff640 1 -- 172.21.15.49:0/2116647660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975401e070 con 0x55b6459abd30 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f976671b640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 0x7f97480a4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9750005cc0 tx=0x7f975002c0f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9747fff640 1 -- 172.21.15.49:0/2116647660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754019b10 con 0x55b6459abd30 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.612+0000 7f9747fff640 1 -- 172.21.15.49:0/2116647660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400ea70 con 0x55b6459abd30 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 -- 172.21.15.49:0/2116647660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 msgr2=0x55b6459ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9754009c90 tx=0x7f9754011810 comp rx=0 tx=0).stop 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 -- 172.21.15.49:0/2116647660 shutdown_connections 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b6459ae120 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a3150 0x55b6459ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2116647660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 -- 172.21.15.49:0/2116647660 >> 172.21.15.49:0/2116647660 conn(0x55b64599cf60 msgr2=0x55b64599d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 -- 172.21.15.49:0/2116647660 shutdown_connections 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.613+0000 7f976ad6d740 1 -- 172.21.15.49:0/2116647660 wait complete. 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 -- start start 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 Processor -- start 2024-03-16T08:19:47.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b6459aeb20 con 0x55b6459a3150 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6458c2230 con 0x55b6459abd30 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b6459aeca0 con 0x55b6459a2830 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976ad6d740 10 client.0 _ll_register_callbacks cb 0x55b6458fe010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f9765719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f9765719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59170/0 (socket says 172.21.15.49:59170) 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f9765719640 1 -- 172.21.15.49:0/422042849 learned_addr learned my addr 172.21.15.49:0/422042849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f976671b640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f9765f1a640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.614+0000 7f9765719640 1 -- 172.21.15.49:0/422042849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 msgr2=0x55b6459a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f9765719640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f9765719640 1 -- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 msgr2=0x55b6459a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f9765719640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f9765719640 1 -- 172.21.15.49:0/422042849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6458d1100 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f9765719640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f975401c040 tx=0x7f9754019a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7860717c25f5ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975401e050 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9754005250 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754019b10 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9754005c50 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f976ad6d740 10 client.16311 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b64561a3f0 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754019b10 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f975400ec60 con 0x55b6459a3150 2024-03-16T08:19:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f976ad6d740 10 client.16311 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.615+0000 7f976ad6d740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f976ad6d740 20 client.16311 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b6459bbf80 0x55b6459be370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f976ad6d740 4 client.16311 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6459139d0 con 0x55b6459bbf80 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f9765f1a640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b6459bbf80 0x55b6459be370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f97457fa640 10 client.16311 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.616+0000 7f9765f1a640 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b6459bbf80 0x55b6459be370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40795498395d84ab server_cookie=c7586e76e019eeb8 in_seq=0 out_seq=0 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f97457fa640 1 -- 172.21.15.49:0/422042849 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b6459139d0 con 0x55b6459bbf80 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f97457fa640 10 client.16311 handle_command_reply: tid=0 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 client.16311 shutdown 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 msgr2=0x55b6459a9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f975401c040 tx=0x7f9754019a30 comp rx=0 tx=0).stop 2024-03-16T08:19:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 shutdown_connections 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b6459bbf80 0x55b6459be370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f97457fa640 0 client.16311 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a3150 0x55b6459a9290 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 --2- 172.21.15.49:0/422042849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a2830 0x55b6459a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 >> 172.21.15.49:0/422042849 conn(0x55b64599cf60 msgr2=0x55b64599ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 shutdown_connections 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 1 -- 172.21.15.49:0/422042849 wait complete. 2024-03-16T08:19:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.617+0000 7f976ad6d740 20 client.16311 trim_cache size 0 max 16384 2024-03-16T08:19:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 Processor -- start 2024-03-16T08:19:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 -- start start 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 0x55b6459ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b1ec0 con 0x55b6459abd30 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97601af790 con 0x55b6459a2830 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.630+0000 7f976ad6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97601ad5c0 con 0x55b6459a3150 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f9765719640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f9765719640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:60018/0 (socket says 172.21.15.49:60018) 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 0x55b6459ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59176/0 (socket says 172.21.15.49:59176) 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f9765719640 1 -- 172.21.15.49:0/866765021 learned_addr learned my addr 172.21.15.49:0/866765021 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 -- 172.21.15.49:0/866765021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 msgr2=0x55b6459b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 -- 172.21.15.49:0/866765021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 msgr2=0x55b6459ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 0x55b6459ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 -- 172.21.15.49:0/866765021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6456e47d0 con 0x55b6459abd30 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f9765f1a640 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 0x55b6459ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f976671b640 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f975002c630 tx=0x7f9750005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60555e9988178fee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f97477fe640 1 -- 172.21.15.49:0/866765021 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750039040 con 0x55b6459abd30 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f97477fe640 1 -- 172.21.15.49:0/866765021 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750054070 con 0x55b6459abd30 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.631+0000 7f97477fe640 1 -- 172.21.15.49:0/866765021 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750004850 con 0x55b6459abd30 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- 172.21.15.49:0/866765021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 msgr2=0x55b64599eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f975002c630 tx=0x7f9750005f70 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- 172.21.15.49:0/866765021 shutdown_connections 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459abd30 0x55b64599eaa0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 --2- 172.21.15.49:0/866765021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459a2830 0x55b6459ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- 172.21.15.49:0/866765021 >> 172.21.15.49:0/866765021 conn(0x55b64599cf60 msgr2=0x55b64599e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- 172.21.15.49:0/866765021 shutdown_connections 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- 172.21.15.49:0/866765021 wait complete. 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 -- start start 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.632+0000 7f976ad6d740 1 Processor -- start 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97601adc50 con 0x55b6459a2830 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97601af0c0 con 0x55b6459abd30 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97601ae320 con 0x55b6459a3150 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976ad6d740 10 client.0 _ll_register_callbacks cb 0x7f975401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59190/0 (socket says 172.21.15.49:59190) 2024-03-16T08:19:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 -- 172.21.15.49:0/2997883888 learned_addr learned my addr 172.21.15.49:0/2997883888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f976671b640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765719640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 -- 172.21.15.49:0/2997883888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 msgr2=0x55b6459a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 -- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 msgr2=0x55b6459a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 -- 172.21.15.49:0/2997883888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b645a21130 con 0x55b6459a2830 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765719640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.633+0000 7f9765f1a640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f975809b770 tx=0x7f975809b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd273bf24144c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:47.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580af060 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f97580860b0 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9750054070 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f975809f930 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 10 client.16317 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b645a22370 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97580b3030 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f97580a7020 con 0x55b6459a2830 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 10 client.16317 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 20 client.16317 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b645a22a20 0x55b645a24e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 4 client.16317 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:19:47.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.634+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9750054070 con 0x55b645a22a20 2024-03-16T08:19:47.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.636+0000 7f9765719640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b645a22a20 0x55b645a24e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:47.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.638+0000 7f97457fa640 10 client.16317 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:47.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.638+0000 7f9765719640 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b645a22a20 0x55b645a24e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef5b36d0dd902b1 server_cookie=851981278678dffd in_seq=0 out_seq=0 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.639+0000 7f97457fa640 1 -- 172.21.15.49:0/2997883888 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9750054070 con 0x55b645a22a20 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.639+0000 7f97457fa640 10 client.16317 handle_command_reply: tid=0 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.639+0000 7f976ad6d740 1 client.16317 shutdown 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.640+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 msgr2=0x55b6459a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.640+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f975809b770 tx=0x7f975809b6e0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.640+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 shutdown_connections 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.640+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b645a22a20 0x55b645a24e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f97457fa640 0 client.16317 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6459abd30 0x55b6459a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b6459a3150 0x55b6459a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 --2- 172.21.15.49:0/2997883888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b6459a2830 0x55b6459a5840 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 >> 172.21.15.49:0/2997883888 conn(0x55b64599cf60 msgr2=0x55b64599ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 shutdown_connections 2024-03-16T08:19:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 1 -- 172.21.15.49:0/2997883888 wait complete. 2024-03-16T08:19:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.641+0000 7f976ad6d740 20 client.16317 trim_cache size 0 max 16384 2024-03-16T08:19:47.642 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:19:47.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 msgr2=0x7f97480a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 0x7f97480a4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9750005cc0 tx=0x7f975002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 msgr2=0x7f97600fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f975800e9f0 tx=0x7f975800edf0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 shutdown_connections 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f97480a29f0 0x7f97480a4eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9760106710 0x7f97600fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97601010c0 0x7f97600fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 --2- 172.21.15.49:0/168664047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9760100700 0x7f97600fd680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.642+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 >> 172.21.15.49:0/168664047 conn(0x7f9760077840 msgr2=0x7f976006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.643+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 shutdown_connections 2024-03-16T08:19:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:47.643+0000 7f976817e640 1 -- 172.21.15.49:0/168664047 wait complete. 2024-03-16T08:19:48.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:48 smithi049 ceph-mon[31942]: pgmap v80: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-03-16T08:19:48.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:48 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1543593193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:48.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:48 smithi129 ceph-mon[36561]: pgmap v80: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-03-16T08:19:48.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:48 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1543593193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:48 smithi154 ceph-mon[36471]: pgmap v80: 97 pgs: 97 active+clean; 515 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-03-16T08:19:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:48 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1543593193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:19:48.659 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:19:48.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:19:48.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:19:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 -- 172.21.15.49:0/392257373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 msgr2=0x7ffbe810d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 --2- 172.21.15.49:0/392257373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe810d540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009930 tx=0x7ffbe002bc30 comp rx=0 tx=0).stop 2024-03-16T08:19:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 -- 172.21.15.49:0/392257373 shutdown_connections 2024-03-16T08:19:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 --2- 172.21.15.49:0/392257373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe810d540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 --2- 172.21.15.49:0/392257373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 0x7ffbe8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 --2- 172.21.15.49:0/392257373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 0x7ffbe8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 -- 172.21.15.49:0/392257373 >> 172.21.15.49:0/392257373 conn(0x7ffbe80777b0 msgr2=0x7ffbe8077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 -- 172.21.15.49:0/392257373 shutdown_connections 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 -- 172.21.15.49:0/392257373 wait complete. 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.792+0000 7ffbef997640 1 Processor -- start 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 -- start start 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 0x7ffbe8196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 0x7ffbe819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8110a90 con 0x7ffbe8101030 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8110910 con 0x7ffbe8106680 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbef997640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8110c10 con 0x7ffbe8100670 2024-03-16T08:19:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59200/0 (socket says 172.21.15.49:59200) 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 -- 172.21.15.49:0/1903042331 learned_addr learned my addr 172.21.15.49:0/1903042331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbedf34640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 0x7ffbe819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbed733640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 0x7ffbe8196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 -- 172.21.15.49:0/1903042331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 msgr2=0x7ffbe8196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 0x7ffbe8196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 -- 172.21.15.49:0/1903042331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 msgr2=0x7ffbe819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 0x7ffbe819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 -- 172.21.15.49:0/1903042331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819bf20 con 0x7ffbe8101030 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.793+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800e9f0 tx=0x7ffbd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.794+0000 7ffbde7fc640 1 -- 172.21.15.49:0/1903042331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800cd80 con 0x7ffbe8101030 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.794+0000 7ffbde7fc640 1 -- 172.21.15.49:0/1903042331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd8010910 con 0x7ffbe8101030 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.794+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe819c210 con 0x7ffbe8101030 2024-03-16T08:19:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.794+0000 7ffbde7fc640 1 -- 172.21.15.49:0/1903042331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8018990 con 0x7ffbe8101030 2024-03-16T08:19:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.794+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a3aa0 con 0x7ffbe8101030 2024-03-16T08:19:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.796+0000 7ffbde7fc640 1 -- 172.21.15.49:0/1903042331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ffbd8010ac0 con 0x7ffbe8101030 2024-03-16T08:19:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.796+0000 7ffbf2585740 1 Processor -- start 2024-03-16T08:19:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.796+0000 7ffbf2585740 1 -- start start 2024-03-16T08:19:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b32710 0x560db6b31460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbde7fc640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 0x7ffbd40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b3cfb0 0x560db6b3f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560db6a950d0 con 0x560db6b3cfb0 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560db6a3dc70 con 0x560db6b32710 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbf2585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560db6a91170 con 0x560db6b31a30 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbed733640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 0x7ffbd40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbedf34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b3cfb0 0x560db6b3f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:60040/0 (socket says 172.21.15.49:60040) 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbde7fc640 1 -- 172.21.15.49:0/1903042331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffbd80cf0b0 con 0x7ffbe8101030 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 -- 172.21.15.49:0/1419867802 learned_addr learned my addr 172.21.15.49:0/1419867802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbedf34640 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b32710 0x560db6b31460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 -- 172.21.15.49:0/1419867802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b32710 msgr2=0x560db6b31460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b32710 0x560db6b31460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 -- 172.21.15.49:0/1419867802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b3cfb0 msgr2=0x560db6b3f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b3cfb0 0x560db6b3f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 -- 172.21.15.49:0/1419867802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db68717e0 con 0x560db6b31a30 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbed733640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 0x7ffbd40a4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4005fd0 tx=0x7ffbe4009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.797+0000 7ffbecf32640 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80cc8b0 tx=0x7ffbd801ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb169dc4a76da7c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbd37fe640 1 -- 172.21.15.49:0/1419867802 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8026040 con 0x560db6b31a30 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbd37fe640 1 -- 172.21.15.49:0/1419867802 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd80adf30 con 0x560db6b31a30 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbd37fe640 1 -- 172.21.15.49:0/1419867802 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8086110 con 0x560db6b31a30 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 -- 172.21.15.49:0/1419867802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 msgr2=0x560db6b31eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80cc8b0 tx=0x7ffbd801ccc0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 -- 172.21.15.49:0/1419867802 shutdown_connections 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b3cfb0 0x560db6b3f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b31a30 0x560db6b31eb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 --2- 172.21.15.49:0/1419867802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b32710 0x560db6b31460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 -- 172.21.15.49:0/1419867802 >> 172.21.15.49:0/1419867802 conn(0x560db6af8c50 msgr2=0x560db6b2c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 -- 172.21.15.49:0/1419867802 shutdown_connections 2024-03-16T08:19:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.798+0000 7ffbf2585740 1 -- 172.21.15.49:0/1419867802 wait complete. 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 -- start start 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 Processor -- start 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 0x560db6b39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 0x560db6b3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b3cfb0 0x560db6b3a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560db6b3fd60 con 0x560db6b31a30 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560db6a510d0 con 0x560db6b3cfb0 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560db6b3fee0 con 0x560db6b32710 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbf2585740 10 client.0 _ll_register_callbacks cb 0x560db6a1a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbed733640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 0x560db6b39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.799+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 0x560db6b3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbed733640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 0x560db6b39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59218/0 (socket says 172.21.15.49:59218) 2024-03-16T08:19:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbed733640 1 -- 172.21.15.49:0/3110652450 learned_addr learned my addr 172.21.15.49:0/3110652450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbedf34640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b3cfb0 0x560db6b3a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 -- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b3cfb0 msgr2=0x560db6b3a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b3cfb0 0x560db6b3a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 -- 172.21.15.49:0/3110652450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 msgr2=0x560db6b39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 0x560db6b39f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 -- 172.21.15.49:0/3110652450 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db68d1f70 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 0x560db6b3a440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80cc8b0 tx=0x7ffbd8099d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95d1b6ed9f3cce5f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80ccad0 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffbd80ccc70 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd80adf30 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffbd80ad150 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbf2585740 10 client.26212 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.800+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560db6769170 con 0x560db6b32710 2024-03-16T08:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd80adf30 con 0x560db6b32710 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffbd8018b30 con 0x560db6b32710 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 10 client.26212 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 20 client.26212 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6b4ce70 0x560db6b4f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 4 client.26212 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db6aa2610 con 0x560db6b4ce70 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbed733640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6b4ce70 0x560db6b4f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbd0ff9640 10 client.26212 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.801+0000 7ffbed733640 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6b4ce70 0x560db6b4f260 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3467e53c507f34e server_cookie=4fb68083ee860a85 in_seq=0 out_seq=0 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/3110652450 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560db6aa2610 con 0x560db6b4ce70 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbd0ff9640 10 client.26212 handle_command_reply: tid=0 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 client.26212 shutdown 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 msgr2=0x560db6b3a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 0x560db6b3a440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80cc8b0 tx=0x7ffbd8099d20 comp rx=0 tx=0).stop 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 shutdown_connections 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6b4ce70 0x560db6b4f260 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b3cfb0 0x560db6b3a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b32710 0x560db6b3a440 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3110652450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b31a30 0x560db6b39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbd0ff9640 0 client.26212 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 >> 172.21.15.49:0/3110652450 conn(0x560db6af8c50 msgr2=0x560db6b2e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 shutdown_connections 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 1 -- 172.21.15.49:0/3110652450 wait complete. 2024-03-16T08:19:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.802+0000 7ffbf2585740 20 client.26212 trim_cache size 0 max 16384 2024-03-16T08:19:48.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 Processor -- start 2024-03-16T08:19:48.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 -- start start 2024-03-16T08:19:48.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6b37180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ada90 con 0x560db6b32710 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ab360 con 0x560db6b31a30 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbf2585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81a91b0 con 0x560db6b3cfb0 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59224/0 (socket says 172.21.15.49:59224) 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.820+0000 7ffbecf32640 1 -- 172.21.15.49:0/3070433966 learned_addr learned my addr 172.21.15.49:0/3070433966 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbed733640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbedf34640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6b37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 -- 172.21.15.49:0/3070433966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 msgr2=0x560db6b37180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6b37180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 -- 172.21.15.49:0/3070433966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 msgr2=0x560db6b3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b3c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 -- 172.21.15.49:0/3070433966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db6b30a60 con 0x560db6b32710 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbed733640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbecf32640 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800e9c0 tx=0x7ffbd80973f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68140c31d80a8dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbd2ffd640 1 -- 172.21.15.49:0/3070433966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd809ac60 con 0x560db6b32710 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbd2ffd640 1 -- 172.21.15.49:0/3070433966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd809ae00 con 0x560db6b32710 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.821+0000 7ffbd2ffd640 1 -- 172.21.15.49:0/3070433966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80afda0 con 0x560db6b32710 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- 172.21.15.49:0/3070433966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 msgr2=0x560db6b48910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800e9c0 tx=0x7ffbd80973f0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- 172.21.15.49:0/3070433966 shutdown_connections 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6b37180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b48910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- 172.21.15.49:0/3070433966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- 172.21.15.49:0/3070433966 >> 172.21.15.49:0/3070433966 conn(0x560db6af8c50 msgr2=0x560db6b48200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- 172.21.15.49:0/3070433966 shutdown_connections 2024-03-16T08:19:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- 172.21.15.49:0/3070433966 wait complete. 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 -- start start 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 Processor -- start 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.822+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6bb4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81a9820 con 0x560db6b32710 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81aac90 con 0x560db6b31a30 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81a9ef0 con 0x560db6b3cfb0 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 10 client.0 _ll_register_callbacks cb 0x7ffbd8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59230/0 (socket says 172.21.15.49:59230) 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 -- 172.21.15.49:0/4013145378 learned_addr learned my addr 172.21.15.49:0/4013145378 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbed733640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b4e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbedf34640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6bb4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 -- 172.21.15.49:0/4013145378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 msgr2=0x560db6bb4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6bb4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 -- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 msgr2=0x560db6b4e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b4e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 -- 172.21.15.49:0/4013145378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db6bb4740 con 0x560db6b32710 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbed733640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbecf32640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbd809b3f0 tx=0x7ffbd8010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320fc61f8af21a41 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd809ac60 con 0x560db6b32710 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ffbd80afd50 con 0x560db6b32710 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560db6a9c960 con 0x560db6b32710 2024-03-16T08:19:48.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffbd80a3f00 con 0x560db6b32710 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 10 client.16335 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.823+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560db6bb4740 con 0x560db6b32710 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560db6a9c960 con 0x560db6b32710 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffbd80a8030 con 0x560db6b32710 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 10 client.16335 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 20 client.16335 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6bb5dd0 0x560db6bb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 4 client.16335 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560db6a1abe0 con 0x560db6bb5dd0 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbed733640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6bb5dd0 0x560db6bb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbed733640 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6bb5dd0 0x560db6bb81c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4276615e4a0c89f1 server_cookie=caaf87826918955 in_seq=0 out_seq=0 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.824+0000 7ffbd0ff9640 10 client.16335 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbd0ff9640 1 -- 172.21.15.49:0/4013145378 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560db6a1abe0 con 0x560db6bb5dd0 2024-03-16T08:19:48.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbd0ff9640 10 client.16335 handle_command_reply: tid=0 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbf2585740 1 client.16335 shutdown 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 msgr2=0x560db6b4e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbd809b3f0 tx=0x7ffbd8010660 comp rx=0 tx=0).stop 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 shutdown_connections 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.825+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560db6bb5dd0 0x560db6bb81c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db6b3cfb0 0x560db6bb4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560db6b32710 0x560db6b4e860 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 --2- 172.21.15.49:0/4013145378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560db6b31a30 0x560db6b4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 >> 172.21.15.49:0/4013145378 conn(0x560db6af8c50 msgr2=0x560db6b2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbd0ff9640 0 client.16335 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 shutdown_connections 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 1 -- 172.21.15.49:0/4013145378 wait complete. 2024-03-16T08:19:48.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbf2585740 20 client.16335 trim_cache size 0 max 16384 2024-03-16T08:19:48.833 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:19:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "c98d6ee8-aa82-40c8-9864-a70bee804ec8", 2024-03-16T08:19:48.833 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:19:48.833 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 msgr2=0x7ffbd40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 0x7ffbd40a4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4005fd0 tx=0x7ffbe4009500 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 msgr2=0x7ffbe8197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800e9f0 tx=0x7ffbd800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 shutdown_connections 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbd40a29f0 0x7ffbd40a4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbe8106680 0x7ffbe819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe8101030 0x7ffbe8197470 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 --2- 172.21.15.49:0/1903042331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe8100670 0x7ffbe8196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 >> 172.21.15.49:0/1903042331 conn(0x7ffbe80777b0 msgr2=0x7ffbe8102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 shutdown_connections 2024-03-16T08:19:48.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.826+0000 7ffbef997640 1 -- 172.21.15.49:0/1903042331 wait complete. 2024-03-16T08:19:48.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:19:48.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 -- 172.21.15.49:0/3362435803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc102a50 msgr2=0x7f5ddc102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 --2- 172.21.15.49:0/3362435803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc102a50 0x7f5ddc102e30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009900 tx=0x7f5dd802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 -- 172.21.15.49:0/3362435803 shutdown_connections 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 --2- 172.21.15.49:0/3362435803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc10be20 0x7f5ddc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 --2- 172.21.15.49:0/3362435803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 --2- 172.21.15.49:0/3362435803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc102a50 0x7f5ddc102e30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 -- 172.21.15.49:0/3362435803 >> 172.21.15.49:0/3362435803 conn(0x7f5ddc077840 msgr2=0x7f5ddc077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 -- 172.21.15.49:0/3362435803 shutdown_connections 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.950+0000 7f5de3590640 1 -- 172.21.15.49:0/3362435803 wait complete. 2024-03-16T08:19:48.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 Processor -- start 2024-03-16T08:19:48.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 -- start start 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 0x7f5ddc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc111bf0 con 0x7f5ddc10be20 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc111a70 con 0x7f5ddc102a50 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de3590640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc111d70 con 0x7f5ddc103370 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de1b2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 0x7f5ddc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de1b2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59248/0 (socket says 172.21.15.49:59248) 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de1b2d640 1 -- 172.21.15.49:0/1567399038 learned_addr learned my addr 172.21.15.49:0/1567399038 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 -- 172.21.15.49:0/1567399038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 msgr2=0x7f5ddc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 -- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 msgr2=0x7f5ddc19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de132c640 1 -- 172.21.15.49:0/1567399038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc19d080 con 0x7f5ddc102a50 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.951+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:48.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5de132c640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 0x7f5ddc198090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8000c00 tx=0x7f5dd802cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5dd27fc640 1 -- 172.21.15.49:0/1567399038 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8041070 con 0x7f5ddc102a50 2024-03-16T08:19:48.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc19d310 con 0x7f5ddc102a50 2024-03-16T08:19:48.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5dd27fc640 1 -- 172.21.15.49:0/1567399038 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd80054e0 con 0x7f5ddc102a50 2024-03-16T08:19:48.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5dd27fc640 1 -- 172.21.15.49:0/1567399038 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003d00 con 0x7f5ddc102a50 2024-03-16T08:19:48.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.952+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a4c00 con 0x7f5ddc102a50 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.954+0000 7f5dd27fc640 1 -- 172.21.15.49:0/1567399038 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5dd8044070 con 0x7f5ddc102a50 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.954+0000 7f5de6178740 1 Processor -- start 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.954+0000 7f5de6178740 1 -- start start 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be85530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562b1bde9500 con 0x562b1be85a70 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b1bd91be0 con 0x562b1be8e650 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de6178740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562b1bde55a0 con 0x562b1be85150 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59250/0 (socket says 172.21.15.49:59250) 2024-03-16T08:19:48.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 -- 172.21.15.49:0/1845124454 learned_addr learned my addr 172.21.15.49:0/1845124454 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de1b2d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be85530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5dd27fc640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 0x7f5dc80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 -- 172.21.15.49:0/1845124454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 msgr2=0x562b1be85530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be85530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 -- 172.21.15.49:0/1845124454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 msgr2=0x562b1be90a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be90a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de132c640 1 -- 172.21.15.49:0/1845124454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1bda54b0 con 0x562b1be85a70 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be85530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be90a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 0x7f5dc80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.955+0000 7f5dd27fc640 1 -- 172.21.15.49:0/1567399038 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5dd80f05b0 con 0x7f5ddc102a50 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.956+0000 7f5de132c640 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80edda0 tx=0x7f5dd804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f13b2cf30335704 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.956+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 0x7f5dc80a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005c40 tx=0x7f5dd4004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.956+0000 7f5dc77fe640 1 -- 172.21.15.49:0/1845124454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd804b070 con 0x562b1be85a70 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.956+0000 7f5dc77fe640 1 -- 172.21.15.49:0/1845124454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd8005690 con 0x562b1be85a70 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.956+0000 7f5dc77fe640 1 -- 172.21.15.49:0/1845124454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80a7ae0 con 0x562b1be85a70 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- 172.21.15.49:0/1845124454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 msgr2=0x562b1be8e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80edda0 tx=0x7f5dd804d000 comp rx=0 tx=0).stop 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- 172.21.15.49:0/1845124454 shutdown_connections 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be90a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8e110 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 --2- 172.21.15.49:0/1845124454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be85530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- 172.21.15.49:0/1845124454 >> 172.21.15.49:0/1845124454 conn(0x562b1be7f880 msgr2=0x562b1be7fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- 172.21.15.49:0/1845124454 shutdown_connections 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- 172.21.15.49:0/1845124454 wait complete. 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 -- start start 2024-03-16T08:19:48.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.957+0000 7f5de6178740 1 Processor -- start 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be85150 0x562b1be8b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be8e650 0x562b1be8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562b1be91200 con 0x562b1be85a70 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b1bda5160 con 0x562b1be85150 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562b1be91380 con 0x562b1be8e650 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 10 client.0 _ll_register_callbacks cb 0x562b1bd89150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59262/0 (socket says 172.21.15.49:59262) 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 -- 172.21.15.49:0/2621057346 learned_addr learned my addr 172.21.15.49:0/2621057346 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de132c640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be85150 0x562b1be8b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be8e650 0x562b1be8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 -- 172.21.15.49:0/2621057346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be8e650 msgr2=0x562b1be8c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be8e650 0x562b1be8c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 -- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be85150 msgr2=0x562b1be8b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be85150 0x562b1be8b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 -- 172.21.15.49:0/2621057346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1bbc77d0 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc012d00 tx=0x7f5dcc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dd9158b16e00e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc009d90 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5dcc0120d0 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.958+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b1bdf6a40 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5dcc004030 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 10 client.16347 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562b1bafd3f0 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b1bdf6a40 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5dcc003c90 con 0x562b1be85a70 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 10 client.16347 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:48.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 20 client.16347 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1be9e290 0x562b1bea0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 4 client.16347 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b1bdf0d90 con 0x562b1be9e290 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.959+0000 7f5de132c640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1be9e290 0x562b1bea0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5dc4ff9640 10 client.16347 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de132c640 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1be9e290 0x562b1bea0680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e55c8f20ab3780 server_cookie=f3937c01e11aaa90 in_seq=0 out_seq=0 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/2621057346 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562b1bdf0d90 con 0x562b1be9e290 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5dc4ff9640 10 client.16347 handle_command_reply: tid=0 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 client.16347 shutdown 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 msgr2=0x562b1be8bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc012d00 tx=0x7f5dcc00f790 comp rx=0 tx=0).stop 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 shutdown_connections 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1be9e290 0x562b1bea0680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be8e650 0x562b1be8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8bb80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5dc4ff9640 0 client.16347 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 --2- 172.21.15.49:0/2621057346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be85150 0x562b1be8b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.960+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 >> 172.21.15.49:0/2621057346 conn(0x562b1be7f880 msgr2=0x562b1be815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.961+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 shutdown_connections 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.961+0000 7f5de6178740 1 -- 172.21.15.49:0/2621057346 wait complete. 2024-03-16T08:19:48.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.961+0000 7f5de6178740 20 client.16347 trim_cache size 0 max 16384 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 Processor -- start 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 -- start start 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1afc80 con 0x562b1be85a70 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ad550 con 0x562b1be8e650 2024-03-16T08:19:48.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de6178740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ab380 con 0x562b1be85150 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de0b2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de0b2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59272/0 (socket says 172.21.15.49:59272) 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de0b2b640 1 -- 172.21.15.49:0/1798547995 learned_addr learned my addr 172.21.15.49:0/1798547995 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.978+0000 7f5de132c640 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 -- 172.21.15.49:0/1798547995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 msgr2=0x562b1be8dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be8dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 -- 172.21.15.49:0/1798547995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 msgr2=0x562b1be86b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be86b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 -- 172.21.15.49:0/1798547995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1bea09f0 con 0x562b1be85a70 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc026500 tx=0x7f5dcc009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df9227f86ad9487 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5dc6ffd640 1 -- 172.21.15.49:0/1798547995 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0120d0 con 0x562b1be85a70 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5dc6ffd640 1 -- 172.21.15.49:0/1798547995 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dcc005730 con 0x562b1be85a70 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5dc6ffd640 1 -- 172.21.15.49:0/1798547995 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0035f0 con 0x562b1be85a70 2024-03-16T08:19:48.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 -- 172.21.15.49:0/1798547995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 msgr2=0x562b1be8a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc026500 tx=0x7f5dcc009f20 comp rx=0 tx=0).stop 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 -- 172.21.15.49:0/1798547995 shutdown_connections 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85a70 0x562b1be8a580 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 --2- 172.21.15.49:0/1798547995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85150 0x562b1be8dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 -- 172.21.15.49:0/1798547995 >> 172.21.15.49:0/1798547995 conn(0x562b1be7f880 msgr2=0x562b1be7e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.979+0000 7f5de6178740 1 -- 172.21.15.49:0/1798547995 shutdown_connections 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 -- 172.21.15.49:0/1798547995 wait complete. 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 -- start start 2024-03-16T08:19:48.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 Processor -- start 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85a70 0x562b1be918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1aba10 con 0x562b1be85150 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ace80 con 0x562b1be8e650 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ac0e0 con 0x562b1be85a70 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de6178740 10 client.0 _ll_register_callbacks cb 0x7f5dcc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de132c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de132c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59280/0 (socket says 172.21.15.49:59280) 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de132c640 1 -- 172.21.15.49:0/1576243053 learned_addr learned my addr 172.21.15.49:0/1576243053 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85a70 0x562b1be918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.980+0000 7f5de1b2d640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 -- 172.21.15.49:0/1576243053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85a70 msgr2=0x562b1be918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85a70 0x562b1be918a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 -- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 msgr2=0x562b1be91de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be91de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 -- 172.21.15.49:0/1576243053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1babd0f0 con 0x562b1be85150 2024-03-16T08:19:48.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de132c640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009900 tx=0x7f5dd80037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97be3e83b6fe055a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd804b040 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5dd80bb120 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b1bd6e540 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80da050 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de6178740 10 client.16353 fetch_fsmap learned FSMap version 24 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562b1bdb09d0 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b1bd6e540 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.981+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5dd80b9490 con 0x562b1be85150 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 10 client.16353 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 20 client.16353 populate_metadata read hostname 'smithi049' 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1bf043a0 0x562b1bf06790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:19:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 4 client.16353 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b1bdbae30 con 0x562b1bf043a0 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1bf043a0 0x562b1bf06790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5dc4ff9640 10 client.16353 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de0b2b640 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1bf043a0 0x562b1bf06790 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f746f690cb4bde server_cookie=77d7ccb69224536d in_seq=0 out_seq=0 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5dc4ff9640 1 -- 172.21.15.49:0/1576243053 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b1bdbae30 con 0x562b1bf043a0 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5dc4ff9640 10 client.16353 handle_command_reply: tid=0 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 client.16353 shutdown 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 msgr2=0x562b1be96530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009900 tx=0x7f5dd80037d0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 shutdown_connections 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562b1bf043a0 0x562b1bf06790 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b1be8e650 0x562b1be91de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b1be85a70 0x562b1be918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5dc4ff9640 0 client.16353 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 --2- 172.21.15.49:0/1576243053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562b1be85150 0x562b1be96530 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.982+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 >> 172.21.15.49:0/1576243053 conn(0x562b1be7f880 msgr2=0x562b1be80bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 shutdown_connections 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de6178740 1 -- 172.21.15.49:0/1576243053 wait complete. 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de6178740 20 client.16353 trim_cache size 0 max 16384 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:19:48.991 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: "c98d6ee8-aa82-40c8-9864-a70bee804ec8": { 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "c98d6ee8-aa82-40c8-9864-a70bee804ec8", 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:19:48.992 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 msgr2=0x7f5dc80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 0x7f5dc80a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005c40 tx=0x7f5dd4004a40 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 msgr2=0x7f5ddc198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 0x7f5ddc198090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8000c00 tx=0x7f5dd802cb50 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 shutdown_connections 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5dc80a2970 0x7f5dc80a4e30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ddc10be20 0x7f5ddc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ddc103370 0x7f5ddc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 --2- 172.21.15.49:0/1567399038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ddc102a50 0x7f5ddc198090 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 >> 172.21.15.49:0/1567399038 conn(0x7f5ddc077840 msgr2=0x7f5ddc0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:19:48.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 shutdown_connections 2024-03-16T08:19:48.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:19:48.983+0000 7f5de3590640 1 -- 172.21.15.49:0/1567399038 wait complete. 2024-03-16T08:19:48.995 INFO:tasks.cephfs.filesystem:scrub status for tag:c98d6ee8-aa82-40c8-9864-a70bee804ec8 - {'path': '/', 'tag': 'c98d6ee8-aa82-40c8-9864-a70bee804ec8', 'options': 'recursive,force'} 2024-03-16T08:19:50.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:50 smithi049 ceph-mon[31942]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-03-16T08:19:50.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:50 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:19:50.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:50 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:50.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:50 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:19:50.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:50 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:50 smithi129 ceph-mon[36561]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-03-16T08:19:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:50 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:19:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:50 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:50 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:19:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:50 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:50.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:50 smithi154 ceph-mon[36471]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-03-16T08:19:50.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:50 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:19:50.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:50 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:50.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:50 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:19:50.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:50 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:19:52.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:52 smithi049 ceph-mon[31942]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:52.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:52 smithi129 ceph-mon[36561]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:52.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:52 smithi154 ceph-mon[36471]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:53.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:53 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:19:53.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:53 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:19:53.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:53 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:19:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:54 smithi049 ceph-mon[31942]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:54 smithi129 ceph-mon[36561]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:54 smithi154 ceph-mon[36471]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-03-16T08:19:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:56 smithi049 ceph-mon[31942]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2024-03-16T08:19:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:56 smithi129 ceph-mon[36561]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2024-03-16T08:19:56.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:56 smithi154 ceph-mon[36471]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2024-03-16T08:19:58.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:19:58 smithi049 ceph-mon[31942]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-16T08:19:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:19:58 smithi129 ceph-mon[36561]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-16T08:19:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:19:58 smithi154 ceph-mon[36471]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-16T08:20:00.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:00 smithi049 ceph-mon[31942]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-03-16T08:20:00.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:00 smithi049 ceph-mon[31942]: overall HEALTH_OK 2024-03-16T08:20:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:00 smithi129 ceph-mon[36561]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-03-16T08:20:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:00 smithi129 ceph-mon[36561]: overall HEALTH_OK 2024-03-16T08:20:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:00 smithi154 ceph-mon[36471]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-03-16T08:20:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:00 smithi154 ceph-mon[36471]: overall HEALTH_OK 2024-03-16T08:20:02.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:02 smithi049 ceph-mon[31942]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:02 smithi129 ceph-mon[36561]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:02 smithi154 ceph-mon[36471]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:04 smithi049 ceph-mon[31942]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:04 smithi129 ceph-mon[36561]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:04 smithi154 ceph-mon[36471]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-16T08:20:06.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:06 smithi049 ceph-mon[31942]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-16T08:20:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:06 smithi129 ceph-mon[36561]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-16T08:20:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:06 smithi154 ceph-mon[36471]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-16T08:20:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:08 smithi049 ceph-mon[31942]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-16T08:20:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:08 smithi129 ceph-mon[36561]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-16T08:20:08.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:08 smithi154 ceph-mon[36471]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-16T08:20:10.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:10 smithi049 ceph-mon[31942]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:20:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:10 smithi129 ceph-mon[36561]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:20:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:10 smithi154 ceph-mon[36471]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-16T08:20:12.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:12 smithi049 ceph-mon[31942]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:12 smithi129 ceph-mon[36561]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:12 smithi154 ceph-mon[36471]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:14.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:14 smithi129 ceph-mon[36561]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:14.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:14 smithi154 ceph-mon[36471]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:14 smithi049 ceph-mon[31942]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:20:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:16 smithi129 ceph-mon[36561]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:20:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:16 smithi154 ceph-mon[36471]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:20:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:16 smithi049 ceph-mon[31942]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:20:18.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:18 smithi129 ceph-mon[36561]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:20:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:18 smithi154 ceph-mon[36471]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:20:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:18 smithi049 ceph-mon[31942]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:20:18.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:20:18.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 -- 172.21.15.49:0/95197707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070102140 msgr2=0x7f707010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 --2- 172.21.15.49:0/95197707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070102140 0x7f707010e650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7068009be0 tx=0x7f706802bef0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 -- 172.21.15.49:0/95197707 shutdown_connections 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 --2- 172.21.15.49:0/95197707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070102140 0x7f707010e650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 --2- 172.21.15.49:0/95197707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070101780 0x7f7070101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 --2- 172.21.15.49:0/95197707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 0x7f7070107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 -- 172.21.15.49:0/95197707 >> 172.21.15.49:0/95197707 conn(0x7f70700fd210 msgr2=0x7f70700ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 -- 172.21.15.49:0/95197707 shutdown_connections 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.120+0000 7f70766c6640 1 -- 172.21.15.49:0/95197707 wait complete. 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 Processor -- start 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 -- start start 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 0x7f707019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7070111ba0 con 0x7f7070102140 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7070111a20 con 0x7f7070101780 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f70766c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7070111d20 con 0x7f7070107790 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f706ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48218/0 (socket says 172.21.15.49:48218) 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f706f7fe640 1 -- 172.21.15.49:0/530630723 learned_addr learned my addr 172.21.15.49:0/530630723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f706ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:45700/0 (socket says 172.21.15.49:45700) 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.121+0000 7f7074c63640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 0x7f707019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 -- 172.21.15.49:0/530630723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 msgr2=0x7f707019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 0x7f707019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 -- 172.21.15.49:0/530630723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 msgr2=0x7f7070198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 -- 172.21.15.49:0/530630723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707019cfc0 con 0x7f7070102140 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706ffff640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706f7fe640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9c0 tx=0x7f706400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706d7fa640 1 -- 172.21.15.49:0/530630723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706400ccc0 con 0x7f7070102140 2024-03-16T08:20:19.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707019d2b0 con 0x7f7070102140 2024-03-16T08:20:19.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706d7fa640 1 -- 172.21.15.49:0/530630723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7064005500 con 0x7f7070102140 2024-03-16T08:20:19.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f706d7fa640 1 -- 172.21.15.49:0/530630723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064010430 con 0x7f7070102140 2024-03-16T08:20:19.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.122+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70701a4b40 con 0x7f7070102140 2024-03-16T08:20:19.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.125+0000 7f706d7fa640 1 -- 172.21.15.49:0/530630723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f70640046c0 con 0x7f7070102140 2024-03-16T08:20:19.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.125+0000 7f707938f740 1 Processor -- start 2024-03-16T08:20:19.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.125+0000 7f707938f740 1 -- start start 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.125+0000 7f707938f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 0x556fcac2e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f707938f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 0x556fcac37110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f707938f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f707938f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556fcab92500 con 0x556fcac37650 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f707938f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556fcab3abe0 con 0x556fcac2e150 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f707938f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556fcab8e5a0 con 0x556fcac2ea70 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f7074c63640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 0x556fcac2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac39a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48232/0 (socket says 172.21.15.49:48232) 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706f7fe640 1 -- 172.21.15.49:0/2645385492 learned_addr learned my addr 172.21.15.49:0/2645385492 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 0x556fcac37110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706d7fa640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 0x7f70580a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 -- 172.21.15.49:0/2645385492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 msgr2=0x556fcac2e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 0x556fcac2e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 -- 172.21.15.49:0/2645385492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 msgr2=0x556fcac39a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac39a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 -- 172.21.15.49:0/2645385492 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fcab4e4b0 con 0x556fcac2ea70 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f7074c63640 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 0x556fcac2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f7074c63640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 0x7f70580a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706d7fa640 1 -- 172.21.15.49:0/530630723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f70640cf0f0 con 0x7f7070102140 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.126+0000 7f706ffff640 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 0x556fcac37110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7060009f50 tx=0x7f70600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46120e2d7eb776d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f70567fc640 1 -- 172.21.15.49:0/2645385492 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060004030 con 0x556fcac2ea70 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f7074c63640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 0x7f70580a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f7068009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f70567fc640 1 -- 172.21.15.49:0/2645385492 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060005500 con 0x556fcac2ea70 2024-03-16T08:20:19.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f70567fc640 1 -- 172.21.15.49:0/2645385492 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060004db0 con 0x556fcac2ea70 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 -- 172.21.15.49:0/2645385492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 msgr2=0x556fcac37110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 0x556fcac37110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7060009f50 tx=0x7f70600118d0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 -- 172.21.15.49:0/2645385492 shutdown_connections 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2ea70 0x556fcac37110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 --2- 172.21.15.49:0/2645385492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2e150 0x556fcac2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 -- 172.21.15.49:0/2645385492 >> 172.21.15.49:0/2645385492 conn(0x556fcac28880 msgr2=0x556fcac28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 -- 172.21.15.49:0/2645385492 shutdown_connections 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.127+0000 7f707938f740 1 -- 172.21.15.49:0/2645385492 wait complete. 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 -- start start 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 Processor -- start 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556fcac3a440 con 0x556fcac37650 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556fcab4e160 con 0x556fcac2ea70 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556fcac3a5c0 con 0x556fcac2e150 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f707938f740 10 client.0 _ll_register_callbacks cb 0x556fcab32150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f7074c63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f706ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f706ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56054/0 (socket says 172.21.15.49:56054) 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f7074c63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48246/0 (socket says 172.21.15.49:48246) 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.128+0000 7f706ffff640 1 -- 172.21.15.49:0/1457382616 learned_addr learned my addr 172.21.15.49:0/1457382616 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 -- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 msgr2=0x556fcac34b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac34b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 -- 172.21.15.49:0/1457382616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 msgr2=0x556fcac350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 -- 172.21.15.49:0/1457382616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fca9707d0 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f706ffff640 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7060012fd0 tx=0x7f70600051a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1297f9302471ec99 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f704f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060029020 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f706000f750 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7060005500 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f70600046c0 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f707938f740 10 client.26248 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556fca8a63f0 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.129+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060005500 con 0x556fcac2e150 2024-03-16T08:20:19.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7060025710 con 0x556fcac2e150 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 10 client.26248 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 20 client.26248 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcac475f0 0x556fcac499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 4 client.26248 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fcab9fa40 con 0x556fcac475f0 2024-03-16T08:20:19.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f706f7fe640 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcac475f0 0x556fcac499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f706f7fe640 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcac475f0 0x556fcac499e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fb44c8c4d524b0 server_cookie=f2306619c6409c14 in_seq=0 out_seq=0 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.130+0000 7f704f7fe640 10 client.26248 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f704f7fe640 1 -- 172.21.15.49:0/1457382616 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556fcab9fa40 con 0x556fcac475f0 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f704f7fe640 10 client.26248 handle_command_reply: tid=0 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 client.26248 shutdown 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 msgr2=0x556fcac34630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7060012fd0 tx=0x7f70600051a0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 shutdown_connections 2024-03-16T08:20:19.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcac475f0 0x556fcac499e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f704f7fe640 0 client.26248 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac34b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 --2- 172.21.15.49:0/1457382616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac34630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.131+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 >> 172.21.15.49:0/1457382616 conn(0x556fcac28880 msgr2=0x556fcac2a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.132+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 shutdown_connections 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.132+0000 7f707938f740 1 -- 172.21.15.49:0/1457382616 wait complete. 2024-03-16T08:20:19.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.132+0000 7f707938f740 20 client.26248 trim_cache size 0 max 16384 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 Processor -- start 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 -- start start 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac37650 0x556fcac4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f70701afd80 con 0x556fcac2e150 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70701ad650 con 0x556fcac2ea70 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f707938f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f70701ab480 con 0x556fcac37650 2024-03-16T08:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f706ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f706ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48260/0 (socket says 172.21.15.49:48260) 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.149+0000 7f706ffff640 1 -- 172.21.15.49:0/3367398602 learned_addr learned my addr 172.21.15.49:0/3367398602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706f7fe640 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f7074c63640 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac37650 0x556fcac4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 -- 172.21.15.49:0/3367398602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac37650 msgr2=0x556fcac4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac37650 0x556fcac4f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 -- 172.21.15.49:0/3367398602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 msgr2=0x556fcac431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 -- 172.21.15.49:0/3367398602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fca8660f0 con 0x556fcac2e150 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f706ffff640 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7060025b80 tx=0x7f70600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da23384793fe4b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f7055ffb640 1 -- 172.21.15.49:0/3367398602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060029040 con 0x556fcac2e150 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f7055ffb640 1 -- 172.21.15.49:0/3367398602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060009b50 con 0x556fcac2e150 2024-03-16T08:20:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.150+0000 7f7055ffb640 1 -- 172.21.15.49:0/3367398602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000f9b0 con 0x556fcac2e150 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- 172.21.15.49:0/3367398602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 msgr2=0x556fcac36d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7060025b80 tx=0x7f70600118d0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- 172.21.15.49:0/3367398602 shutdown_connections 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac37650 0x556fcac4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- 172.21.15.49:0/3367398602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac2e150 0x556fcac36d60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- 172.21.15.49:0/3367398602 >> 172.21.15.49:0/3367398602 conn(0x556fcac28880 msgr2=0x556fcac345f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- 172.21.15.49:0/3367398602 shutdown_connections 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- 172.21.15.49:0/3367398602 wait complete. 2024-03-16T08:20:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- start start 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 Processor -- start 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f70701abb10 con 0x556fcac37650 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70701acf80 con 0x556fcac2ea70 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.151+0000 7f707938f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f70701ac1e0 con 0x556fcac2e150 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f707938f740 10 client.0 _ll_register_callbacks cb 0x7f7064097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:45746/0 (socket says 172.21.15.49:45746) 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56066/0 (socket says 172.21.15.49:56066) 2024-03-16T08:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48270/0 (socket says 172.21.15.49:48270) 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706f7fe640 1 -- 172.21.15.49:0/1242258910 learned_addr learned my addr 172.21.15.49:0/1242258910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 -- 172.21.15.49:0/1242258910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 msgr2=0x556fcac312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 -- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 msgr2=0x556fcac31820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 -- 172.21.15.49:0/1242258910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fcab599d0 con 0x556fcac37650 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706f7fe640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f706ffff640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f7074c63640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7068033420 tx=0x7f7068005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bfed2f01dae7a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f704f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068034900 con 0x556fcac37650 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7068034aa0 con 0x556fcac37650 2024-03-16T08:20:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7060009b50 con 0x556fcac37650 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7068034ca0 con 0x556fcac37650 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f707938f740 10 client.16371 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.152+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556fcacad8a0 con 0x556fcac37650 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060009b50 con 0x556fcac37650 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7068055020 con 0x556fcac37650 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 10 client.16371 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 20 client.16371 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcacadf70 0x556fcacb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 4 client.16371 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fcab99d90 con 0x556fcacadf70 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f706ffff640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcacadf70 0x556fcacb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f704f7fe640 10 client.16371 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.153+0000 7f706ffff640 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcacadf70 0x556fcacb0360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f710a44ceee9870 server_cookie=d7280f83318833b3 in_seq=0 out_seq=0 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f704f7fe640 1 -- 172.21.15.49:0/1242258910 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556fcab99d90 con 0x556fcacadf70 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f704f7fe640 10 client.16371 handle_command_reply: tid=0 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 client.16371 shutdown 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 msgr2=0x556fcac31d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7068033420 tx=0x7f7068005bd0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 shutdown_connections 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556fcacadf70 0x556fcacb0360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fcac37650 0x556fcac31d60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fcac2ea70 0x556fcac31820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f704f7fe640 0 client.16371 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 --2- 172.21.15.49:0/1242258910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556fcac2e150 0x556fcac312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 >> 172.21.15.49:0/1242258910 conn(0x556fcac28880 msgr2=0x556fcac27900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 shutdown_connections 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 1 -- 172.21.15.49:0/1242258910 wait complete. 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f707938f740 20 client.16371 trim_cache size 0 max 16384 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:20:19.163 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:19.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 msgr2=0x7f70580a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 0x7f70580a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f7068009500 comp rx=0 tx=0).stop 2024-03-16T08:20:19.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 msgr2=0x7f7070198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.154+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9c0 tx=0x7f706400ee80 comp rx=0 tx=0).stop 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 shutdown_connections 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f70580a2970 0x7f70580a4e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070107790 0x7f707019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102140 0x7f7070198560 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 --2- 172.21.15.49:0/530630723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7070101780 0x7f7070198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 >> 172.21.15.49:0/530630723 conn(0x7f70700fd210 msgr2=0x7f70700fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 shutdown_connections 2024-03-16T08:20:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.155+0000 7f70766c6640 1 -- 172.21.15.49:0/530630723 wait complete. 2024-03-16T08:20:19.166 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:20:19.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:20:19.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.275+0000 7f57df259640 1 -- 172.21.15.49:0/2102121488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8101770 msgr2=0x7f57d8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.275+0000 7f57df259640 1 --2- 172.21.15.49:0/2102121488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8101770 0x7f57d8101bf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009930 tx=0x7f57cc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.275+0000 7f57df259640 1 -- 172.21.15.49:0/2102121488 shutdown_connections 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.275+0000 7f57df259640 1 --2- 172.21.15.49:0/2102121488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8102130 0x7f57d810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 --2- 172.21.15.49:0/2102121488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8101770 0x7f57d8101bf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 --2- 172.21.15.49:0/2102121488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8107780 0x7f57d8107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- 172.21.15.49:0/2102121488 >> 172.21.15.49:0/2102121488 conn(0x7f57d80fd280 msgr2=0x7f57d80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- 172.21.15.49:0/2102121488 shutdown_connections 2024-03-16T08:20:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- 172.21.15.49:0/2102121488 wait complete. 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 Processor -- start 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- start start 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 0x7f57d8073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 0x7f57d8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8111be0 con 0x7f57d8102130 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8111a60 con 0x7f57d8107780 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57df259640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8111d60 con 0x7f57d8101770 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.276+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48292/0 (socket says 172.21.15.49:48292) 2024-03-16T08:20:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 -- 172.21.15.49:0/1454468025 learned_addr learned my addr 172.21.15.49:0/1454468025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57dcff5640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 0x7f57d8073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57dd7f6640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 0x7f57d8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 -- 172.21.15.49:0/1454468025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 msgr2=0x7f57d8073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 0x7f57d8073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 -- 172.21.15.49:0/1454468025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 msgr2=0x7f57d8074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 0x7f57d8074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 -- 172.21.15.49:0/1454468025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81a6580 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57dcff5640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 0x7f57d8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57dd7f6640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 0x7f57d8074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d7fff640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc043020 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57cc0054e0 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81a6810 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc003b50 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.277+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a6d60 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.278+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8068a20 con 0x7f57d8102130 2024-03-16T08:20:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.279+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f57cc046020 con 0x7f57d8102130 2024-03-16T08:20:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.280+0000 7f57d5ffb640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 0x7f57c00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.280+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f57cc0f0c70 con 0x7f57d8102130 2024-03-16T08:20:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.280+0000 7f57dcff5640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 0x7f57c00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.281+0000 7f57dcff5640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 0x7f57c00a4e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009f40 tx=0x7f57c8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.285+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57cc04d020 con 0x7f57d8102130 2024-03-16T08:20:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.506+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57d8109ce0 con 0x7f57d8102130 2024-03-16T08:20:19.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.507+0000 7f57d5ffb640 1 -- 172.21.15.49:0/1454468025 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f57cc0bdb30 con 0x7f57d8102130 2024-03-16T08:20:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:20:19.507 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:20:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 msgr2=0x7f57c00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 0x7f57c00a4e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009f40 tx=0x7f57c8005f90 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 msgr2=0x7f57d8074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc004360 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 shutdown_connections 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57c00a29a0 0x7f57c00a4e60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8107780 0x7f57d8074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d8102130 0x7f57d8074020 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 --2- 172.21.15.49:0/1454468025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57d8101770 0x7f57d8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 >> 172.21.15.49:0/1454468025 conn(0x7f57d80fd280 msgr2=0x7f57d80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 shutdown_connections 2024-03-16T08:20:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.509+0000 7f57df259640 1 -- 172.21.15.49:0/1454468025 wait complete. 2024-03-16T08:20:19.521 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:20:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 -- 172.21.15.49:0/427187368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c103370 msgr2=0x7fd22c10b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 --2- 172.21.15.49:0/427187368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c103370 0x7fd22c10b850 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 -- 172.21.15.49:0/427187368 shutdown_connections 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 --2- 172.21.15.49:0/427187368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 0x7fd22c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 --2- 172.21.15.49:0/427187368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c103370 0x7fd22c10b850 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 --2- 172.21.15.49:0/427187368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c102a50 0x7fd22c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 -- 172.21.15.49:0/427187368 >> 172.21.15.49:0/427187368 conn(0x7fd22c077840 msgr2=0x7fd22c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 -- 172.21.15.49:0/427187368 shutdown_connections 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.631+0000 7fd233285640 1 -- 172.21.15.49:0/427187368 wait complete. 2024-03-16T08:20:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 Processor -- start 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 -- start start 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c103370 0x7fd22c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 0x7fd22c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c111bf0 con 0x7fd22c102a50 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c111a70 con 0x7fd22c103370 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd233285640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c111d70 con 0x7fd22c10be20 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48308/0 (socket says 172.21.15.49:48308) 2024-03-16T08:20:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 -- 172.21.15.49:0/647801960 learned_addr learned my addr 172.21.15.49:0/647801960 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231822640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 0x7fd22c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd230820640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c103370 0x7fd22c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 -- 172.21.15.49:0/647801960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 msgr2=0x7fd22c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 0x7fd22c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 -- 172.21.15.49:0/647801960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c103370 msgr2=0x7fd22c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c103370 0x7fd22c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.632+0000 7fd231021640 1 -- 172.21.15.49:0/647801960 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c19d0b0 con 0x7fd22c102a50 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd231021640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd22800ea60 tx=0x7fd22800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd2227fc640 1 -- 172.21.15.49:0/647801960 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22800cde0 con 0x7fd22c102a50 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd2227fc640 1 -- 172.21.15.49:0/647801960 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2280035f0 con 0x7fd22c102a50 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c19d3a0 con 0x7fd22c102a50 2024-03-16T08:20:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd2227fc640 1 -- 172.21.15.49:0/647801960 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228010670 con 0x7fd22c102a50 2024-03-16T08:20:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.633+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c1a4c30 con 0x7fd22c102a50 2024-03-16T08:20:19.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.635+0000 7fd2227fc640 1 -- 172.21.15.49:0/647801960 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd2280037a0 con 0x7fd22c102a50 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 Processor -- start 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 -- start start 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a00167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0019120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55679ff71490 con 0x5567a000d830 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55679ff19c90 con 0x5567a000e150 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd235e79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55679ff6d530 con 0x5567a0016d30 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd2227fc640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 0x7fd2180a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd231822640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd231021640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a00167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd230820640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0019120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd231822640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48322/0 (socket says 172.21.15.49:48322) 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd231822640 1 -- 172.21.15.49:0/3244833143 learned_addr learned my addr 172.21.15.49:0/3244833143 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.636+0000 7fd231822640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 0x7fd2180a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd2227fc640 1 -- 172.21.15.49:0/647801960 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd2280cf0d0 con 0x7fd22c102a50 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 -- 172.21.15.49:0/3244833143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 msgr2=0x5567a00167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a00167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 -- 172.21.15.49:0/3244833143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 msgr2=0x5567a000dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 -- 172.21.15.49:0/3244833143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679ffd4500 con 0x5567a0016d30 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd231822640 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd231822640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 0x7fd2180a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd2240099e0 tx=0x7fd224004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd230820640 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0019120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd21c000c00 tx=0x7fd21c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd09b03d1a340d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd2177fe640 1 -- 172.21.15.49:0/3244833143 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c042070 con 0x5567a0016d30 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd2177fe640 1 -- 172.21.15.49:0/3244833143 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c004040 con 0x5567a0016d30 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.637+0000 7fd2177fe640 1 -- 172.21.15.49:0/3244833143 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c004cf0 con 0x5567a0016d30 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- 172.21.15.49:0/3244833143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 msgr2=0x5567a0019120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0019120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd21c000c00 tx=0x7fd21c009560 comp rx=0 tx=0).stop 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- 172.21.15.49:0/3244833143 shutdown_connections 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0019120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a00167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 --2- 172.21.15.49:0/3244833143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a000dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- 172.21.15.49:0/3244833143 >> 172.21.15.49:0/3244833143 conn(0x5567a0007f60 msgr2=0x5567a0008340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- 172.21.15.49:0/3244833143 shutdown_connections 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- 172.21.15.49:0/3244833143 wait complete. 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 -- start start 2024-03-16T08:20:19.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.638+0000 7fd235e79740 1 Processor -- start 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 0x5567a0014250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a0016d30 0x5567a0014790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5567a0019ac0 con 0x5567a000d830 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55679ff2d230 con 0x5567a0016d30 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5567a0019c40 con 0x5567a000e150 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd235e79740 10 client.0 _ll_register_callbacks cb 0x55679ff69010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48334/0 (socket says 172.21.15.49:48334) 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 -- 172.21.15.49:0/4191899666 learned_addr learned my addr 172.21.15.49:0/4191899666 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd230820640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 0x5567a0014250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231822640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a0016d30 0x5567a0014790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 -- 172.21.15.49:0/4191899666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 msgr2=0x5567a0014250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 0x5567a0014250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 -- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a0016d30 msgr2=0x5567a0014790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a0016d30 0x5567a0014790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 -- 172.21.15.49:0/4191899666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679ff3c100 con 0x5567a000d830 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd230820640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 0x5567a0014250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd231021640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd22800d010 tx=0x7fd22806a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48423128c280553 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.639+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22809be40 con 0x5567a000d830 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd228097db0 con 0x5567a000d830 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c004040 con 0x5567a000d830 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd22809b740 con 0x5567a000d830 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 10 client.16389 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55679fc853f0 con 0x5567a000d830 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c004040 con 0x5567a000d830 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd22806a020 con 0x5567a000d830 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 10 client.16389 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 10 client.16389 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 20 client.16389 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a0027000 0x5567a0029430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 4 client.16389 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55679ff7e9d0 con 0x5567a0027000 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.640+0000 7fd230820640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a0027000 0x5567a0029430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd214ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd230820640 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a0027000 0x5567a0029430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ab58be8ff15062d server_cookie=1390d90cc6f0ba7b in_seq=0 out_seq=0 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd214ff9640 1 -- 172.21.15.49:0/4191899666 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55679ff7e9d0 con 0x5567a0027000 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd214ff9640 10 client.16389 handle_command_reply: tid=0 2024-03-16T08:20:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 client.16389 shutdown 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 msgr2=0x5567a0013d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd22800d010 tx=0x7fd22806a550 comp rx=0 tx=0).stop 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 shutdown_connections 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a0027000 0x5567a0029430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a0016d30 0x5567a0014790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd214ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000e150 0x5567a0014250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.641+0000 7fd235e79740 1 --2- 172.21.15.49:0/4191899666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000d830 0x5567a0013d10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.642+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 >> 172.21.15.49:0/4191899666 conn(0x5567a0007f60 msgr2=0x5567a0009cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.642+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 shutdown_connections 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.642+0000 7fd235e79740 1 -- 172.21.15.49:0/4191899666 wait complete. 2024-03-16T08:20:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.642+0000 7fd235e79740 20 client.16389 trim_cache size 0 max 16384 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 Processor -- start 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 -- start start 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000d830 0x5567a0016440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0012e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1afc30 con 0x5567a000e150 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ad500 con 0x5567a000d830 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd235e79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ab330 con 0x5567a0016d30 2024-03-16T08:20:19.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd230820640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd230820640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48338/0 (socket says 172.21.15.49:48338) 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd230820640 1 -- 172.21.15.49:0/3109438041 learned_addr learned my addr 172.21.15.49:0/3109438041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd231021640 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000d830 0x5567a0016440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.655+0000 7fd231822640 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0012e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 -- 172.21.15.49:0/3109438041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 msgr2=0x5567a0012e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0012e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 -- 172.21.15.49:0/3109438041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000d830 msgr2=0x5567a0016440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000d830 0x5567a0016440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 -- 172.21.15.49:0/3109438041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679fd4f7d0 con 0x5567a000e150 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd230820640 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5567a00090f0 tx=0x7fd21c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae6d9f2a8452626 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd216ffd640 1 -- 172.21.15.49:0/3109438041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c04c040 con 0x5567a000e150 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd216ffd640 1 -- 172.21.15.49:0/3109438041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c02c920 con 0x5567a000e150 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd216ffd640 1 -- 172.21.15.49:0/3109438041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0519c0 con 0x5567a000e150 2024-03-16T08:20:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd235e79740 1 -- 172.21.15.49:0/3109438041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 msgr2=0x5567a0022860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.656+0000 7fd235e79740 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5567a00090f0 tx=0x7fd21c0048d0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- 172.21.15.49:0/3109438041 shutdown_connections 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a0016d30 0x5567a0012e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a000e150 0x5567a0022860 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- 172.21.15.49:0/3109438041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000d830 0x5567a0016440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- 172.21.15.49:0/3109438041 >> 172.21.15.49:0/3109438041 conn(0x5567a0007f60 msgr2=0x5567a0006750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- 172.21.15.49:0/3109438041 shutdown_connections 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- 172.21.15.49:0/3109438041 wait complete. 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- start start 2024-03-16T08:20:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 Processor -- start 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 0x5567a001ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 0x5567a0019ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ab9c0 con 0x5567a0016d30 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ace30 con 0x5567a000e150 2024-03-16T08:20:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ac090 con 0x5567a000d830 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.657+0000 7fd235e79740 10 client.0 _ll_register_callbacks cb 0x7fd22800e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231021640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 0x5567a001ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 0x5567a0019ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd230820640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd230820640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:45814/0 (socket says 172.21.15.49:45814) 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231021640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 0x5567a001ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56124/0 (socket says 172.21.15.49:56124) 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd230820640 1 -- 172.21.15.49:0/1640844581 learned_addr learned my addr 172.21.15.49:0/1640844581 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 -- 172.21.15.49:0/1640844581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 msgr2=0x5567a001ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 0x5567a001ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 -- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 msgr2=0x5567a001efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 -- 172.21.15.49:0/1640844581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a001a680 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd230820640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd231822640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 0x5567a0019ef0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd224013640 tx=0x7fd224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da55a4516fdaa481 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240047a0 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd224004b20 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c02c920 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd224004d20 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd235e79740 10 client.16395 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.658+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5567a008cdd0 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c02c920 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd224022040 con 0x5567a0016d30 2024-03-16T08:20:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 10 client.16395 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 10 client.16395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 20 client.16395 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a008d530 0x5567a008f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 4 client.16395 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55679ff78d20 con 0x5567a008d530 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd231021640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a008d530 0x5567a008f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd214ff9640 10 client.16395 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.659+0000 7fd231021640 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a008d530 0x5567a008f950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5b18888be32081b server_cookie=2f0c44bc60b9078c in_seq=0 out_seq=0 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd214ff9640 1 -- 172.21.15.49:0/1640844581 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55679ff78d20 con 0x5567a008d530 2024-03-16T08:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd214ff9640 10 client.16395 handle_command_reply: tid=0 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 client.16395 shutdown 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 msgr2=0x5567a0019ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 0x5567a0019ef0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd224013640 tx=0x7fd224004a40 comp rx=0 tx=0).stop 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 shutdown_connections 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5567a008d530 0x5567a008f950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567a0016d30 0x5567a0019ef0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5567a000e150 0x5567a001efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 --2- 172.21.15.49:0/1640844581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5567a000d830 0x5567a001ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd214ff9640 0 client.16395 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 >> 172.21.15.49:0/1640844581 conn(0x5567a0007f60 msgr2=0x5567a0009390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 shutdown_connections 2024-03-16T08:20:19.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 1 -- 172.21.15.49:0/1640844581 wait complete. 2024-03-16T08:20:19.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd235e79740 20 client.16395 trim_cache size 0 max 16384 2024-03-16T08:20:19.670 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 msgr2=0x7fd2180a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 0x7fd2180a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd2240099e0 tx=0x7fd224004ab0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 msgr2=0x7fd22c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd22800ea60 tx=0x7fd22800ef20 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 shutdown_connections 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd2180a29f0 0x7fd2180a4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd22c10be20 0x7fd22c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd22c103370 0x7fd22c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 --2- 172.21.15.49:0/647801960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd22c102a50 0x7fd22c198110 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.660+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 >> 172.21.15.49:0/647801960 conn(0x7fd22c077840 msgr2=0x7fd22c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.661+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 shutdown_connections 2024-03-16T08:20:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.661+0000 7fd233285640 1 -- 172.21.15.49:0/647801960 wait complete. 2024-03-16T08:20:19.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:20:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 -- 172.21.15.49:0/1872078330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd01010c0 msgr2=0x7f4dd010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 --2- 172.21.15.49:0/1872078330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd01010c0 0x7f4dd010d5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009be0 tx=0x7f4dc802bef0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 -- 172.21.15.49:0/1872078330 shutdown_connections 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 --2- 172.21.15.49:0/1872078330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd01010c0 0x7f4dd010d5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 --2- 172.21.15.49:0/1872078330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd0100700 0x7f4dd0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 --2- 172.21.15.49:0/1872078330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0106710 0x7f4dd0106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 -- 172.21.15.49:0/1872078330 >> 172.21.15.49:0/1872078330 conn(0x7f4dd0077840 msgr2=0x7f4dd0077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 -- 172.21.15.49:0/1872078330 shutdown_connections 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.783+0000 7f4dd8777640 1 -- 172.21.15.49:0/1872078330 wait complete. 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 Processor -- start 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 -- start start 2024-03-16T08:20:19.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0100700 0x7f4dd0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 0x7f4dd019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0110b20 con 0x7f4dd01010c0 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01109a0 con 0x7f4dd0106710 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd8777640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0110ca0 con 0x7f4dd0100700 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd6513640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0100700 0x7f4dd0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48358/0 (socket says 172.21.15.49:48358) 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd6d14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 0x7f4dd019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 -- 172.21.15.49:0/3978507376 learned_addr learned my addr 172.21.15.49:0/3978507376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 -- 172.21.15.49:0/3978507376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0100700 msgr2=0x7f4dd0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0100700 0x7f4dd0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 -- 172.21.15.49:0/3978507376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 msgr2=0x7f4dd019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 0x7f4dd019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 -- 172.21.15.49:0/3978507376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd019d060 con 0x7f4dd01010c0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 0x7f4dd019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.784+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000e9c0 tx=0x7f4dc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.785+0000 7f4dc77fe640 1 -- 172.21.15.49:0/3978507376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000cde0 con 0x7f4dd01010c0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.785+0000 7f4dc77fe640 1 -- 172.21.15.49:0/3978507376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc00035f0 con 0x7f4dd01010c0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.785+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd019d350 con 0x7f4dd01010c0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.785+0000 7f4dc77fe640 1 -- 172.21.15.49:0/3978507376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0010690 con 0x7f4dd01010c0 2024-03-16T08:20:19.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.785+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a4be0 con 0x7f4dd01010c0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.787+0000 7f4dc77fe640 1 -- 172.21.15.49:0/3978507376 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4dc00037a0 con 0x7f4dd01010c0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.787+0000 7f4ddb366740 1 Processor -- start 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.787+0000 7f4ddb366740 1 -- start start 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 0x557c644a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c6448e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c643f7490 con 0x557c6448e030 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c6439fc90 con 0x557c644a1880 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4ddb366740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c643f3530 con 0x557c644a0ed0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c6448e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6d14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 0x557c644a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:45844/0 (socket says 172.21.15.49:45844) 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c6448e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48362/0 (socket says 172.21.15.49:48362) 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6d14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 0x557c644a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56152/0 (socket says 172.21.15.49:56152) 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 -- 172.21.15.49:0/99324677 learned_addr learned my addr 172.21.15.49:0/99324677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dc77fe640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 0x7f4dbc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 0x7f4dbc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 -- 172.21.15.49:0/99324677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 msgr2=0x557c644a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 0x557c644a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 -- 172.21.15.49:0/99324677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 msgr2=0x557c6448e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c6448e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dc77fe640 1 -- 172.21.15.49:0/3978507376 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4dc00cf0b0 con 0x7f4dd01010c0 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 -- 172.21.15.49:0/99324677 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6445a500 con 0x557c644a1880 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6513640 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009c70 tx=0x7f4dcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5062bc31c30ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.788+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 0x7f4dbc0a4e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8006010 tx=0x7f4dc802bef0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4dc4ff9640 1 -- 172.21.15.49:0/99324677 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc004230 con 0x557c644a1880 2024-03-16T08:20:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4dc4ff9640 1 -- 172.21.15.49:0/99324677 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc0043d0 con 0x557c644a1880 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4dc4ff9640 1 -- 172.21.15.49:0/99324677 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc019820 con 0x557c644a1880 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 -- 172.21.15.49:0/99324677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 msgr2=0x557c6448da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009c70 tx=0x7f4dcc011810 comp rx=0 tx=0).stop 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 -- 172.21.15.49:0/99324677 shutdown_connections 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c6448e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c6448da10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 --2- 172.21.15.49:0/99324677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a0ed0 0x557c644a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 -- 172.21.15.49:0/99324677 >> 172.21.15.49:0/99324677 conn(0x557c644904d0 msgr2=0x557c644908b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 -- 172.21.15.49:0/99324677 shutdown_connections 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.789+0000 7f4ddb366740 1 -- 172.21.15.49:0/99324677 wait complete. 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 -- start start 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 Processor -- start 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 0x557c644a5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c644a8310 con 0x557c6448e030 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c643b3230 con 0x557c644a0ed0 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c644a8490 con 0x557c644a1880 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4dd6513640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4ddb366740 10 client.0 _ll_register_callbacks cb 0x557c643ef010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4dd6513640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48376/0 (socket says 172.21.15.49:48376) 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 0x557c644a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4dd6513640 1 -- 172.21.15.49:0/3840722955 learned_addr learned my addr 172.21.15.49:0/3840722955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.790+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 -- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 msgr2=0x557c644a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 -- 172.21.15.49:0/3840722955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 msgr2=0x557c644a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 0x557c644a5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 -- 172.21.15.49:0/3840722955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c643c2100 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 0x557c644a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4dd6513640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc004360 tx=0x7f4dcc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793522a0ba162a1d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc027070 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4dcc003800 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dcc0043d0 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4dcc033050 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4ddb366740 10 client.16407 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c6410b3f0 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc0043d0 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.791+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4dcc018070 con 0x557c6448e030 2024-03-16T08:20:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 10 client.16407 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 10 client.16407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 20 client.16407 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c644b5750 0x557c644b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 4 client.16407 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c644049d0 con 0x557c644b5750 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c644b5750 0x557c644b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c644b5750 0x557c644b7b40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bf879df9c31eda1 server_cookie=bf294a5d630028cd in_seq=0 out_seq=0 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.792+0000 7f4db9ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4db9ffb640 1 -- 172.21.15.49:0/3840722955 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557c644049d0 con 0x557c644b5750 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4db9ffb640 10 client.16407 handle_command_reply: tid=0 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 client.16407 shutdown 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 msgr2=0x557c644a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc004360 tx=0x7f4dcc0056d0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 shutdown_connections 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c644b5750 0x557c644b7b40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a0ed0 0x557c644a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4db9ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 --2- 172.21.15.49:0/3840722955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c6448e030 0x557c644a4e10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 >> 172.21.15.49:0/3840722955 conn(0x557c644904d0 msgr2=0x557c6448fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 shutdown_connections 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 1 -- 172.21.15.49:0/3840722955 wait complete. 2024-03-16T08:20:19.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.793+0000 7f4ddb366740 20 client.16407 trim_cache size 0 max 16384 2024-03-16T08:20:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.806+0000 7f4ddb366740 1 Processor -- start 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.806+0000 7f4ddb366740 1 -- start start 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.806+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 0x557c644a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.806+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4ddb366740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01adb10 con 0x557c644a0ed0 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4ddb366740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01ab3e0 con 0x557c6448e030 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4ddb366740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01a9210 con 0x557c644a1880 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48392/0 (socket says 172.21.15.49:48392) 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 -- 172.21.15.49:0/780448095 learned_addr learned my addr 172.21.15.49:0/780448095 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd6513640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 0x557c644a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 -- 172.21.15.49:0/780448095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 msgr2=0x557c644b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 -- 172.21.15.49:0/780448095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 msgr2=0x557c644a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 0x557c644a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 -- 172.21.15.49:0/780448095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c641d57d0 con 0x557c644a0ed0 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd6513640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 0x557c644a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4dc009b500 tx=0x7f4dc00d1610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d4ab5297260da0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dbbfff640 1 -- 172.21.15.49:0/780448095 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc009b160 con 0x557c644a0ed0 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dbbfff640 1 -- 172.21.15.49:0/780448095 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc009bf00 con 0x557c644a0ed0 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.807+0000 7f4dbbfff640 1 -- 172.21.15.49:0/780448095 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0099b90 con 0x557c644a0ed0 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- 172.21.15.49:0/780448095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 msgr2=0x557c644a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4dc009b500 tx=0x7f4dc00d1610 comp rx=0 tx=0).stop 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- 172.21.15.49:0/780448095 shutdown_connections 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c644a1880 0x557c644b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644a4710 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 --2- 172.21.15.49:0/780448095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6448e030 0x557c644a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- 172.21.15.49:0/780448095 >> 172.21.15.49:0/780448095 conn(0x557c644904d0 msgr2=0x557c6448cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- 172.21.15.49:0/780448095 shutdown_connections 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- 172.21.15.49:0/780448095 wait complete. 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 -- start start 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.808+0000 7f4ddb366740 1 Processor -- start 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6448e030 0x557c644ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c644bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01a98a0 con 0x557c644a0ed0 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01aad10 con 0x557c644a1880 2024-03-16T08:20:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01a9f70 con 0x557c6448e030 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 10 client.0 _ll_register_callbacks cb 0x7f4dc0097b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48396/0 (socket says 172.21.15.49:48396) 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 -- 172.21.15.49:0/1504313139 learned_addr learned my addr 172.21.15.49:0/1504313139 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd6513640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6448e030 0x557c644ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd6d14640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c644bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 -- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6448e030 msgr2=0x557c644ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6448e030 0x557c644ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 -- 172.21.15.49:0/1504313139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 msgr2=0x557c644bc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c644bc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 -- 172.21.15.49:0/1504313139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c644bcd90 con 0x557c644a0ed0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4dd5d12640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0097c60 tx=0x7f4dc009b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c12d38abfd54ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4db9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0084d40 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4dc00860b0 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc009bf00 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4dc009b800 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 10 client.16413 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.809+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c6451b530 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc009bf00 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4dc009cb30 con 0x557c644a0ed0 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 10 client.16413 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 10 client.16413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 20 client.16413 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c6451bb70 0x557c6451df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 4 client.16413 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c643fed20 con 0x557c6451bb70 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4dd6513640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c6451bb70 0x557c6451df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4db9ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4dd6513640 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c6451bb70 0x557c6451df90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9acc902ceceb1e56 server_cookie=d41d5594d3496623 in_seq=0 out_seq=0 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4db9ffb640 1 -- 172.21.15.49:0/1504313139 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c643fed20 con 0x557c6451bb70 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4db9ffb640 10 client.16413 handle_command_reply: tid=0 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.810+0000 7f4ddb366740 1 client.16413 shutdown 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 msgr2=0x557c644bc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0097c60 tx=0x7f4dc009b220 comp rx=0 tx=0).stop 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 shutdown_connections 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x557c6451bb70 0x557c6451df90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c644a1880 0x557c644bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4db9ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c644a0ed0 0x557c644bc310 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 --2- 172.21.15.49:0/1504313139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6448e030 0x557c644ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 >> 172.21.15.49:0/1504313139 conn(0x557c644904d0 msgr2=0x557c644a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 shutdown_connections 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 1 -- 172.21.15.49:0/1504313139 wait complete. 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4ddb366740 20 client.16413 trim_cache size 0 max 16384 2024-03-16T08:20:19.819 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 msgr2=0x7f4dbc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 0x7f4dbc0a4e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8006010 tx=0x7f4dc802bef0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 msgr2=0x7f4dd01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000e9c0 tx=0x7f4dc000ee80 comp rx=0 tx=0).stop 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 shutdown_connections 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4dbc0a2970 0x7f4dbc0a4e30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dd0106710 0x7f4dd019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd01010c0 0x7f4dd01985b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 --2- 172.21.15.49:0/3978507376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd0100700 0x7f4dd0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 >> 172.21.15.49:0/3978507376 conn(0x7f4dd0077840 msgr2=0x7f4dd006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 shutdown_connections 2024-03-16T08:20:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.811+0000 7f4dd8777640 1 -- 172.21.15.49:0/3978507376 wait complete. 2024-03-16T08:20:19.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:20:19.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- 172.21.15.49:0/4186442397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8101010 msgr2=0x7f6ef810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 --2- 172.21.15.49:0/4186442397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8101010 0x7f6ef810d520 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009930 tx=0x7f6ef002bc30 comp rx=0 tx=0).stop 2024-03-16T08:20:19.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- 172.21.15.49:0/4186442397 shutdown_connections 2024-03-16T08:20:19.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 --2- 172.21.15.49:0/4186442397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8101010 0x7f6ef810d520 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 --2- 172.21.15.49:0/4186442397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8100650 0x7f6ef8100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 --2- 172.21.15.49:0/4186442397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 0x7f6ef8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- 172.21.15.49:0/4186442397 >> 172.21.15.49:0/4186442397 conn(0x7f6ef80777e0 msgr2=0x7f6ef8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- 172.21.15.49:0/4186442397 shutdown_connections 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- 172.21.15.49:0/4186442397 wait complete. 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 Processor -- start 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.934+0000 7f6f00259640 1 -- start start 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8101010 0x7f6ef8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 0x7f6ef81a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8110a70 con 0x7f6ef8100650 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81108f0 con 0x7f6ef8101010 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8110bf0 con 0x7f6ef8106660 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48412/0 (socket says 172.21.15.49:48412) 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 -- 172.21.15.49:0/4084626368 learned_addr learned my addr 172.21.15.49:0/4084626368 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8101010 0x7f6ef8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 0x7f6ef81a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 -- 172.21.15.49:0/4084626368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 msgr2=0x7f6ef81a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 0x7f6ef81a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 -- 172.21.15.49:0/4084626368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8101010 msgr2=0x7f6ef8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8101010 0x7f6ef8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 -- 172.21.15.49:0/4084626368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef81a4410 con 0x7f6ef8100650 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e9f0 tx=0x7f6eec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6eeaffd640 1 -- 172.21.15.49:0/4084626368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00cd80 con 0x7f6ef8100650 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6eeaffd640 1 -- 172.21.15.49:0/4084626368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec005500 con 0x7f6ef8100650 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6eeaffd640 1 -- 172.21.15.49:0/4084626368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec010690 con 0x7f6ef8100650 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef81a45e0 con 0x7f6ef8100650 2024-03-16T08:20:19.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.935+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a4ae0 con 0x7f6ef8100650 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6eeaffd640 1 -- 172.21.15.49:0/4084626368 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6eec0046c0 con 0x7f6ef8100650 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 Processor -- start 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 -- start start 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c77c9d490 con 0x555c77d47140 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c77c45c90 con 0x555c77d394e0 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.938+0000 7f6f02e43740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555c77c99530 con 0x555c77d39e00 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48414/0 (socket says 172.21.15.49:48414) 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 -- 172.21.15.49:0/839053004 learned_addr learned my addr 172.21.15.49:0/839053004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efdff5640 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6eeaffd640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 0x7f6ee40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 -- 172.21.15.49:0/839053004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 msgr2=0x555c77d46c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d46c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 -- 172.21.15.49:0/839053004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 msgr2=0x555c77d398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 -- 172.21.15.49:0/839053004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c77d00500 con 0x555c77d47140 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 0x7f6ee40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6eeaffd640 1 -- 172.21.15.49:0/4084626368 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6eec0cf1f0 con 0x7f6ef8100650 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4011470 tx=0x7f6ef4011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678311477dbc4797 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6ee3fff640 1 -- 172.21.15.49:0/839053004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401e070 con 0x555c77d47140 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6ee3fff640 1 -- 172.21.15.49:0/839053004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef4003c30 con 0x555c77d47140 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.939+0000 7f6ee3fff640 1 -- 172.21.15.49:0/839053004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400fa00 con 0x555c77d47140 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- 172.21.15.49:0/839053004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 msgr2=0x555c77d49530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 0x7f6ee40a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0005940 tx=0x7f6ef002e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4011470 tx=0x7f6ef4011930 comp rx=0 tx=0).stop 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- 172.21.15.49:0/839053004 shutdown_connections 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77d49530 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d46c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 --2- 172.21.15.49:0/839053004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- 172.21.15.49:0/839053004 >> 172.21.15.49:0/839053004 conn(0x555c77d33f60 msgr2=0x555c77d34340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- 172.21.15.49:0/839053004 shutdown_connections 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- 172.21.15.49:0/839053004 wait complete. 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.940+0000 7f6f02e43740 1 -- start start 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 Processor -- start 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d39e00 0x555c77d44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d47140 0x555c77d44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c77d49f80 con 0x555c77d394e0 2024-03-16T08:20:19.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c77c59230 con 0x555c77d39e00 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555c77d4a100 con 0x555c77d47140 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6f02e43740 10 client.0 _ll_register_callbacks cb 0x555c77c95010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48418/0 (socket says 172.21.15.49:48418) 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 -- 172.21.15.49:0/3967811040 learned_addr learned my addr 172.21.15.49:0/3967811040 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d39e00 0x555c77d44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d47140 0x555c77d44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 -- 172.21.15.49:0/3967811040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d47140 msgr2=0x555c77d44b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d47140 0x555c77d44b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 -- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d39e00 msgr2=0x555c77d44630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d39e00 0x555c77d44630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.941+0000 7f6efdff5640 1 -- 172.21.15.49:0/3967811040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c77c68100 con 0x555c77d394e0 2024-03-16T08:20:19.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6efdff5640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6eec097d00 tx=0x7f6eec0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f798b195b506e778 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec09b670 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6eec09bb00 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef4003c30 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6eec09b200 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 10 client.16425 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c779b13f0 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef4003c30 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6eec09cbd0 con 0x555c77d394e0 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 10 client.16425 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 20 client.16425 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77d575b0 0x555c77d59a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 4 client.16425 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c77caa9d0 con 0x555c77d575b0 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.942+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77d575b0 0x555c77d59a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.943+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77d575b0 0x555c77d59a00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d184bca37ce9992a server_cookie=388b2a2b8ecf305f in_seq=0 out_seq=0 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.943+0000 7f6ee17fa640 10 client.16425 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.943+0000 7f6ee17fa640 1 -- 172.21.15.49:0/3967811040 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555c77caa9d0 con 0x555c77d575b0 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.943+0000 7f6ee17fa640 10 client.16425 handle_command_reply: tid=0 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 client.16425 shutdown 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 msgr2=0x555c77d440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6eec097d00 tx=0x7f6eec0056b0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 shutdown_connections 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77d575b0 0x555c77d59a00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d47140 0x555c77d44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d39e00 0x555c77d44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 --2- 172.21.15.49:0/3967811040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d440f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6ee17fa640 0 client.16425 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 >> 172.21.15.49:0/3967811040 conn(0x555c77d33f60 msgr2=0x555c77d35cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 shutdown_connections 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 1 -- 172.21.15.49:0/3967811040 wait complete. 2024-03-16T08:20:19.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.944+0000 7f6f02e43740 20 client.16425 trim_cache size 0 max 16384 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 Processor -- start 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 -- start start 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d47140 0x555c77d43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81ada40 con 0x555c77d394e0 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81ab310 con 0x555c77d47140 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6f02e43740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81a9160 con 0x555c77d39e00 2024-03-16T08:20:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6efdff5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48434/0 (socket says 172.21.15.49:48434) 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6efdff5640 1 -- 172.21.15.49:0/4244611670 learned_addr learned my addr 172.21.15.49:0/4244611670 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.957+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d47140 0x555c77d43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 -- 172.21.15.49:0/4244611670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 msgr2=0x555c77d52d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d52d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 -- 172.21.15.49:0/4244611670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d47140 msgr2=0x555c77d43250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d47140 0x555c77d43250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 -- 172.21.15.49:0/4244611670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c77a7b7d0 con 0x555c77d394e0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6efdff5640 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0a4200 tx=0x7f6eec0d2880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedd19fef03c72a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6ee37fe640 1 -- 172.21.15.49:0/4244611670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec09b3e0 con 0x555c77d394e0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6ee37fe640 1 -- 172.21.15.49:0/4244611670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec0b8070 con 0x555c77d394e0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.958+0000 7f6ee37fe640 1 -- 172.21.15.49:0/4244611670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec09b8a0 con 0x555c77d394e0 2024-03-16T08:20:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- 172.21.15.49:0/4244611670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 msgr2=0x555c77d46820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0a4200 tx=0x7f6eec0d2880 comp rx=0 tx=0).stop 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- 172.21.15.49:0/4244611670 shutdown_connections 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d47140 0x555c77d43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d52d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- 172.21.15.49:0/4244611670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d394e0 0x555c77d46820 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- 172.21.15.49:0/4244611670 >> 172.21.15.49:0/4244611670 conn(0x555c77d33f60 msgr2=0x555c77d32750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- 172.21.15.49:0/4244611670 shutdown_connections 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- 172.21.15.49:0/4244611670 wait complete. 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- start start 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 Processor -- start 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d4a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d4aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81a97d0 con 0x555c77d47140 2024-03-16T08:20:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81aac40 con 0x555c77d394e0 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81a9ea0 con 0x555c77d39e00 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.959+0000 7f6f02e43740 10 client.0 _ll_register_callbacks cb 0x7f6eec00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48444/0 (socket says 172.21.15.49:48444) 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 -- 172.21.15.49:0/887904812 learned_addr learned my addr 172.21.15.49:0/887904812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efdff5640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d4a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efd7f4640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d4aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 -- 172.21.15.49:0/887904812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 msgr2=0x555c77d4aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d4aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 -- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 msgr2=0x555c77d4a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d4a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 -- 172.21.15.49:0/887904812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c77dbbe90 con 0x555c77d47140 2024-03-16T08:20:19.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efdff5640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d4a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6efe7f6640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009230 tx=0x7f6ef00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82b92a0a8a3e1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0005110 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6ef00052b0 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eec0b8070 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6ef005b020 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6f02e43740 10 client.16431 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.960+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c77dbd170 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec0b8070 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6ef0034440 con 0x555c77d47140 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 10 client.16431 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 20 client.16431 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77dbd890 0x555c77dbfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 4 client.16431 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c77ca4d20 con 0x555c77dbd890 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6efdff5640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77dbd890 0x555c77dbfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6ee17fa640 10 client.16431 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6efdff5640 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77dbd890 0x555c77dbfc80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e6297979c39200 server_cookie=6ed5a8a7df12727f in_seq=0 out_seq=0 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6ee17fa640 1 -- 172.21.15.49:0/887904812 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c77ca4d20 con 0x555c77dbd890 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6ee17fa640 10 client.16431 handle_command_reply: tid=0 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.961+0000 7f6f02e43740 1 client.16431 shutdown 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 msgr2=0x555c77dbb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009230 tx=0x7f6ef00058d0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 shutdown_connections 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555c77dbd890 0x555c77dbfc80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c77d47140 0x555c77dbb8a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555c77d39e00 0x555c77d4aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 --2- 172.21.15.49:0/887904812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c77d394e0 0x555c77d4a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6ee17fa640 0 client.16431 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 >> 172.21.15.49:0/887904812 conn(0x555c77d33f60 msgr2=0x555c77d440b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 shutdown_connections 2024-03-16T08:20:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 1 -- 172.21.15.49:0/887904812 wait complete. 2024-03-16T08:20:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f02e43740 20 client.16431 trim_cache size 0 max 16384 2024-03-16T08:20:19.969 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 msgr2=0x7f6ee40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 0x7f6ee40a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0005940 tx=0x7f6ef002e040 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 msgr2=0x7f6ef8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e9f0 tx=0x7f6eec00edf0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 shutdown_connections 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6ee40a2970 0x7f6ee40a4e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef8106660 0x7f6ef81a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ef8101010 0x7f6ef8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 --2- 172.21.15.49:0/4084626368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef8100650 0x7f6ef8198e50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 >> 172.21.15.49:0/4084626368 conn(0x7f6ef80777e0 msgr2=0x7f6ef8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 shutdown_connections 2024-03-16T08:20:19.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:19.962+0000 7f6f00259640 1 -- 172.21.15.49:0/4084626368 wait complete. 2024-03-16T08:20:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:20 smithi049 ceph-mon[31942]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:20:20.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:20 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1454468025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:20 smithi129 ceph-mon[36561]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:20:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:20 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1454468025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:20 smithi154 ceph-mon[36471]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:20:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:20 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1454468025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:20.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:20:20.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:20:20.974 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:20:21.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 -- 172.21.15.49:0/2733646360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 msgr2=0x7f745c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 --2- 172.21.15.49:0/2733646360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c10e620 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7454009cd0 tx=0x7f745402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 -- 172.21.15.49:0/2733646360 shutdown_connections 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 --2- 172.21.15.49:0/2733646360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c10e620 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 --2- 172.21.15.49:0/2733646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 --2- 172.21.15.49:0/2733646360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 -- 172.21.15.49:0/2733646360 >> 172.21.15.49:0/2733646360 conn(0x7f745c0777e0 msgr2=0x7f745c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 -- 172.21.15.49:0/2733646360 shutdown_connections 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.090+0000 7f7463ad1640 1 -- 172.21.15.49:0/2733646360 wait complete. 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 Processor -- start 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 -- start start 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f745c111b70 con 0x7f745c069380 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1119f0 con 0x7f745c107760 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f7463ad1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f745c111cf0 con 0x7f745c0689c0 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48460/0 (socket says 172.21.15.49:48460) 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f746106c640 1 -- 172.21.15.49:0/747890871 learned_addr learned my addr 172.21.15.49:0/747890871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f746186d640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.091+0000 7f746206e640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 -- 172.21.15.49:0/747890871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 msgr2=0x7f745c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 -- 172.21.15.49:0/747890871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 msgr2=0x7f745c19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 -- 172.21.15.49:0/747890871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c19d070 con 0x7f745c069380 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746206e640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746186d640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f746106c640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9f0 tx=0x7f745800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f7452ffd640 1 -- 172.21.15.49:0/747890871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800cd80 con 0x7f745c069380 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f7452ffd640 1 -- 172.21.15.49:0/747890871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7458005500 con 0x7f745c069380 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c19d360 con 0x7f745c069380 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f7452ffd640 1 -- 172.21.15.49:0/747890871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458010690 con 0x7f745c069380 2024-03-16T08:20:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.092+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c1a4bf0 con 0x7f745c069380 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7452ffd640 1 -- 172.21.15.49:0/747890871 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f74580046c0 con 0x7f745c069380 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 Processor -- start 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 -- start start 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 0x55f83a6fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f83a64e0d0 con 0x55f83a6eb710 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f83a5f6c70 con 0x55f83a6f81b0 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f7466798740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f83a64a170 con 0x55f83a6eaa30 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f746206e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f746186d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f746106c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 0x55f83a6fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f746206e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6ea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48474/0 (socket says 172.21.15.49:48474) 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.095+0000 7f746206e640 1 -- 172.21.15.49:0/420890725 learned_addr learned my addr 172.21.15.49:0/420890725 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f7452ffd640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 0x7f74480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746206e640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 0x7f74480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 -- 172.21.15.49:0/420890725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 msgr2=0x55f83a6fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 0x55f83a6fa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 -- 172.21.15.49:0/420890725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 msgr2=0x55f83a6ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f7452ffd640 1 -- 172.21.15.49:0/747890871 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f74580cf200 con 0x7f745c069380 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 -- 172.21.15.49:0/420890725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83a42a7e0 con 0x55f83a6eaa30 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746106c640 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 0x55f83a6fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746186d640 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6eaeb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f744c009f50 tx=0x7f744c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6484e5b00ff5243a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f746206e640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 0x7f74480a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f745402c0f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f7447fff640 1 -- 172.21.15.49:0/420890725 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c004030 con 0x55f83a6eaa30 2024-03-16T08:20:21.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f7447fff640 1 -- 172.21.15.49:0/420890725 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c005500 con 0x55f83a6eaa30 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.096+0000 7f7447fff640 1 -- 172.21.15.49:0/420890725 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c004d50 con 0x55f83a6eaa30 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 -- 172.21.15.49:0/420890725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 msgr2=0x55f83a6eaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6eaeb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f744c009f50 tx=0x7f744c0118d0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 -- 172.21.15.49:0/420890725 shutdown_connections 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6f81b0 0x55f83a6fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6eaeb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 --2- 172.21.15.49:0/420890725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 -- 172.21.15.49:0/420890725 >> 172.21.15.49:0/420890725 conn(0x55f83a6b1c50 msgr2=0x55f83a6e5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 -- 172.21.15.49:0/420890725 shutdown_connections 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.097+0000 7f7466798740 1 -- 172.21.15.49:0/420890725 wait complete. 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 -- start start 2024-03-16T08:20:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 Processor -- start 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a6f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f83a6fb020 con 0x55f83a6eb710 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f83a60a0d0 con 0x55f83a6eaa30 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f83a6fb1a0 con 0x55f83a6f81b0 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f7466798740 10 client.0 _ll_register_callbacks cb 0x55f83a5d3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48480/0 (socket says 172.21.15.49:48480) 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f746106c640 1 -- 172.21.15.49:0/315291064 learned_addr learned my addr 172.21.15.49:0/315291064 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f746186d640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a6f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.098+0000 7f746206e640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 -- 172.21.15.49:0/315291064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 msgr2=0x55f83a6f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 -- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 msgr2=0x55f83a6f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a6f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 -- 172.21.15.49:0/315291064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83a48af70 con 0x55f83a6eb710 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f746106c640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9c0 tx=0x7f7458004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b337b6ef50d41b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580862b0 con 0x55f83a6eb710 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f74580ccc20 con 0x55f83a6eb710 2024-03-16T08:20:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c005500 con 0x55f83a6eb710 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f74580cce20 con 0x55f83a6eb710 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f7466798740 10 client.16443 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f83a322170 con 0x55f83a6eb710 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c005500 con 0x55f83a6eb710 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.099+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f74580a3bc0 con 0x55f83a6eb710 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 10 client.16443 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 20 client.16443 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a708240 0x55f83a70a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 4 client.16443 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83a65b610 con 0x55f83a708240 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f746186d640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a708240 0x55f83a70a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f746186d640 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a708240 0x55f83a70a630 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a230d077dedca3 server_cookie=11c096602e9eeb9c in_seq=0 out_seq=0 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.100+0000 7f74457fa640 10 client.16443 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f74457fa640 1 -- 172.21.15.49:0/315291064 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f83a65b610 con 0x55f83a708240 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f74457fa640 10 client.16443 handle_command_reply: tid=0 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 client.16443 shutdown 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 msgr2=0x55f83a6f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9c0 tx=0x7f7458004ae0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 shutdown_connections 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a708240 0x55f83a70a630 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6f5610 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f74457fa640 0 client.16443 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 --2- 172.21.15.49:0/315291064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a6f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.101+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 >> 172.21.15.49:0/315291064 conn(0x55f83a6b1c50 msgr2=0x55f83a6e71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.102+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 shutdown_connections 2024-03-16T08:20:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.102+0000 7f7466798740 1 -- 172.21.15.49:0/315291064 wait complete. 2024-03-16T08:20:21.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.102+0000 7f7466798740 20 client.16443 trim_cache size 0 max 16384 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.119+0000 7f7466798740 1 Processor -- start 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.119+0000 7f7466798740 1 -- start start 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.119+0000 7f7466798740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.119+0000 7f7466798740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.119+0000 7f7466798740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6f81b0 0x55f83a70ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f7466798740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1adb10 con 0x55f83a6f81b0 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f7466798740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ab3e0 con 0x55f83a6eb710 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f7466798740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1a9210 con 0x55f83a6eaa30 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746186d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746206e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6f81b0 0x55f83a70ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:45964/0 (socket says 172.21.15.49:45964) 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746186d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56270/0 (socket says 172.21.15.49:56270) 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 -- 172.21.15.49:0/3146714049 learned_addr learned my addr 172.21.15.49:0/3146714049 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 -- 172.21.15.49:0/3146714049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 msgr2=0x55f83a6f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 -- 172.21.15.49:0/3146714049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6f81b0 msgr2=0x55f83a70ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6f81b0 0x55f83a70ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 -- 172.21.15.49:0/3146714049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83a6f0cb0 con 0x55f83a6eb710 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f746106c640 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f745809b840 tx=0x7f7458099dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9602c170c88fae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f74477fe640 1 -- 172.21.15.49:0/3146714049 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745809cbe0 con 0x55f83a6eb710 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f74477fe640 1 -- 172.21.15.49:0/3146714049 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745809b8b0 con 0x55f83a6eb710 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.120+0000 7f74477fe640 1 -- 172.21.15.49:0/3146714049 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745809b3f0 con 0x55f83a6eb710 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 -- 172.21.15.49:0/3146714049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 msgr2=0x55f83a703e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f745809b840 tx=0x7f7458099dc0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 -- 172.21.15.49:0/3146714049 shutdown_connections 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6f81b0 0x55f83a70ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eb710 0x55f83a703e00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 --2- 172.21.15.49:0/3146714049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6eaa30 0x55f83a6f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 -- 172.21.15.49:0/3146714049 >> 172.21.15.49:0/3146714049 conn(0x55f83a6b1c50 msgr2=0x55f83a6e44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 -- 172.21.15.49:0/3146714049 shutdown_connections 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.121+0000 7f7466798740 1 -- 172.21.15.49:0/3146714049 wait complete. 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 -- start start 2024-03-16T08:20:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 Processor -- start 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a700620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1a98a0 con 0x55f83a6eb710 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aad10 con 0x55f83a6eaa30 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1a9f70 con 0x55f83a6f81b0 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f7466798740 10 client.0 _ll_register_callbacks cb 0x7f745800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f746106c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48506/0 (socket says 172.21.15.49:48506) 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f746106c640 1 -- 172.21.15.49:0/3256020607 learned_addr learned my addr 172.21.15.49:0/3256020607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f746186d640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a700620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.122+0000 7f746206e640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 -- 172.21.15.49:0/3256020607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 msgr2=0x55f83a6fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 -- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 msgr2=0x55f83a700620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a700620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 -- 172.21.15.49:0/3256020607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83a6fe030 con 0x55f83a6eb710 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746186d640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a700620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f746106c640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9c0 tx=0x7f74580cf510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a72973c2885c4e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f74457fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745809b3f0 con 0x55f83a6eb710 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f745809af10 con 0x55f83a6eb710 2024-03-16T08:20:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745809b8b0 con 0x55f83a6eb710 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f745809cbe0 con 0x55f83a6eb710 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f7466798740 10 client.16449 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f83a6fe030 con 0x55f83a6eb710 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.123+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745809b8b0 con 0x55f83a6eb710 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f74580a0920 con 0x55f83a6eb710 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 10 client.16449 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 20 client.16449 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a76f6b0 0x55f83a771aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 4 client.16449 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f83a655960 con 0x55f83a76f6b0 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f746186d640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a76f6b0 0x55f83a771aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f74457fa640 10 client.16449 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.124+0000 7f746186d640 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a76f6b0 0x55f83a771aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23b5e69e67c4ca7 server_cookie=6dadb7e19951446e in_seq=0 out_seq=0 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f74457fa640 1 -- 172.21.15.49:0/3256020607 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f83a655960 con 0x55f83a76f6b0 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f74457fa640 10 client.16449 handle_command_reply: tid=0 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 client.16449 shutdown 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 msgr2=0x55f83a6fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9c0 tx=0x7f74580cf510 comp rx=0 tx=0).stop 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 shutdown_connections 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f83a76f6b0 0x55f83a771aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f83a6f81b0 0x55f83a6fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f83a6eb710 0x55f83a6fd400 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f74457fa640 0 client.16449 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 --2- 172.21.15.49:0/3256020607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f83a6eaa30 0x55f83a700620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 >> 172.21.15.49:0/3256020607 conn(0x55f83a6b1c50 msgr2=0x55f83a6f5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 shutdown_connections 2024-03-16T08:20:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 1 -- 172.21.15.49:0/3256020607 wait complete. 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.126+0000 7f7466798740 20 client.16449 trim_cache size 0 max 16384 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "35eb933e-2349-4bdf-a78f-c3ed55f1edbe", 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:20:21.132 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 msgr2=0x7f74480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 0x7f74480a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f745402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 msgr2=0x7f745c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9f0 tx=0x7f745800edf0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 shutdown_connections 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f74480a2970 0x7f74480a4e30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f745c107760 0x7f745c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f745c069380 0x7f745c198610 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 --2- 172.21.15.49:0/747890871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f745c0689c0 0x7f745c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 >> 172.21.15.49:0/747890871 conn(0x7f745c0777e0 msgr2=0x7f745c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 shutdown_connections 2024-03-16T08:20:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.127+0000 7f7463ad1640 1 -- 172.21.15.49:0/747890871 wait complete. 2024-03-16T08:20:21.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:20:21.138 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:20:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 -- 172.21.15.49:0/2798512776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854106680 msgr2=0x7f5854106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 --2- 172.21.15.49:0/2798512776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854106680 0x7f5854106a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5844009930 tx=0x7f584402bc30 comp rx=0 tx=0).stop 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 -- 172.21.15.49:0/2798512776 shutdown_connections 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 --2- 172.21.15.49:0/2798512776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 0x7f585410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 --2- 172.21.15.49:0/2798512776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854100670 0x7f5854100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 --2- 172.21.15.49:0/2798512776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854106680 0x7f5854106a60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 -- 172.21.15.49:0/2798512776 >> 172.21.15.49:0/2798512776 conn(0x7f58540777b0 msgr2=0x7f5854077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 -- 172.21.15.49:0/2798512776 shutdown_connections 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 -- 172.21.15.49:0/2798512776 wait complete. 2024-03-16T08:20:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.252+0000 7f5859937640 1 Processor -- start 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 -- start start 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 0x7f5854198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854106680 0x7f585419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5854110a90 con 0x7f5854100670 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5854110910 con 0x7f5854101030 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5859937640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5854110c10 con 0x7f5854106680 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48536/0 (socket says 172.21.15.49:48536) 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 -- 172.21.15.49:0/2668817348 learned_addr learned my addr 172.21.15.49:0/2668817348 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f58527fc640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 0x7f5854198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f58537fe640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854106680 0x7f585419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 -- 172.21.15.49:0/2668817348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854106680 msgr2=0x7f585419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854106680 0x7f585419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 -- 172.21.15.49:0/2668817348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 msgr2=0x7f5854198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 0x7f5854198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 -- 172.21.15.49:0/2668817348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585419d0d0 con 0x7f5854100670 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.253+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5844009230 tx=0x7f584402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.254+0000 7f5858935640 1 -- 172.21.15.49:0/2668817348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844041070 con 0x7f5854100670 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.254+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585419d360 con 0x7f5854100670 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.254+0000 7f5858935640 1 -- 172.21.15.49:0/2668817348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5844004450 con 0x7f5854100670 2024-03-16T08:20:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.254+0000 7f5858935640 1 -- 172.21.15.49:0/2668817348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844003de0 con 0x7f5854100670 2024-03-16T08:20:21.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.254+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58541a4c50 con 0x7f5854100670 2024-03-16T08:20:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.256+0000 7f5858935640 1 -- 172.21.15.49:0/2668817348 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5844044070 con 0x7f5854100670 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.256+0000 7f585c5f6740 1 Processor -- start 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.256+0000 7f585c5f6740 1 -- start start 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.256+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 0x5560e80e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80eca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f585c5f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5560e8045500 con 0x5560e80ea650 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f585c5f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5560e7fedbe0 con 0x5560e80e1150 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f585c5f6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5560e80415a0 con 0x5560e80e1a70 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f5858935640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 0x7f583c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 0x5560e80e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 0x5560e80e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:46000/0 (socket says 172.21.15.49:46000) 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58537fe640 1 -- 172.21.15.49:0/1786640872 learned_addr learned my addr 172.21.15.49:0/1786640872 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58537fe640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 0x7f583c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f5858935640 1 -- 172.21.15.49:0/2668817348 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f58440f0640 con 0x7f5854100670 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 -- 172.21.15.49:0/1786640872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 msgr2=0x5560e80ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 -- 172.21.15.49:0/1786640872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 msgr2=0x5560e80e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 0x5560e80e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 -- 172.21.15.49:0/1786640872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e80014b0 con 0x5560e80ea650 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58537fe640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 0x7f583c0a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f584c005c40 tx=0x7f584c009c10 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.257+0000 7f58527fc640 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80eca40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5848009bd0 tx=0x7f5848011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c247b5b65aba0061 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f5839ffb640 1 -- 172.21.15.49:0/1786640872 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848004230 con 0x5560e80ea650 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f5839ffb640 1 -- 172.21.15.49:0/1786640872 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5848009e40 con 0x5560e80ea650 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f5839ffb640 1 -- 172.21.15.49:0/1786640872 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848004d00 con 0x5560e80ea650 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 -- 172.21.15.49:0/1786640872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 msgr2=0x5560e80eca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80eca40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5848009bd0 tx=0x7f5848011810 comp rx=0 tx=0).stop 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 -- 172.21.15.49:0/1786640872 shutdown_connections 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80eca40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 --2- 172.21.15.49:0/1786640872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1150 0x5560e80e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 -- 172.21.15.49:0/1786640872 >> 172.21.15.49:0/1786640872 conn(0x5560e80db880 msgr2=0x5560e80dbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 -- 172.21.15.49:0/1786640872 shutdown_connections 2024-03-16T08:20:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.258+0000 7f585c5f6740 1 -- 172.21.15.49:0/1786640872 wait complete. 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 -- start start 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 Processor -- start 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5560e80ed440 con 0x5560e80ea650 2024-03-16T08:20:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5560e8001160 con 0x5560e80e1a70 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5560e80ed5c0 con 0x5560e80e1150 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.259+0000 7f585c5f6740 10 client.0 _ll_register_callbacks cb 0x5560e7fe5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58527fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48552/0 (socket says 172.21.15.49:48552) 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 -- 172.21.15.49:0/2530427008 learned_addr learned my addr 172.21.15.49:0/2530427008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 -- 172.21.15.49:0/2530427008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 msgr2=0x5560e80e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 -- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 msgr2=0x5560e80e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 -- 172.21.15.49:0/2530427008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e7e237d0 con 0x5560e80ea650 2024-03-16T08:20:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58527fc640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f58537fe640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f584c003e40 tx=0x7f584c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc5b5f19d43eab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f5832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0116a0 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f584c020670 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560e8052a40 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f584c0208b0 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f585c5f6740 10 client.16461 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.260+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5560e7d593f0 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5560e8052a40 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f584c027070 con 0x5560e80ea650 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 10 client.16461 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 20 client.16461 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e80fa670 0x5560e80fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 4 client.16461 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e804cd90 con 0x5560e80fa670 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.261+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e80fa670 0x5560e80fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.262+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e80fa670 0x5560e80fcac0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3147a0bfe5f76d59 server_cookie=b6a9bca2289a7708 in_seq=0 out_seq=0 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.262+0000 7f5832ffd640 10 client.16461 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f5832ffd640 1 -- 172.21.15.49:0/2530427008 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5560e804cd90 con 0x5560e80fa670 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f5832ffd640 10 client.16461 handle_command_reply: tid=0 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 client.16461 shutdown 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 msgr2=0x5560e80e80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f584c003e40 tx=0x7f584c009900 comp rx=0 tx=0).stop 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 shutdown_connections 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e80fa670 0x5560e80fcac0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e80f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f5832ffd640 0 client.16461 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2530427008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 >> 172.21.15.49:0/2530427008 conn(0x5560e80db880 msgr2=0x5560e80dd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 shutdown_connections 2024-03-16T08:20:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 1 -- 172.21.15.49:0/2530427008 wait complete. 2024-03-16T08:20:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.263+0000 7f585c5f6740 20 client.16461 trim_cache size 0 max 16384 2024-03-16T08:20:21.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 Processor -- start 2024-03-16T08:20:21.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 -- start start 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 0x5560e80e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58541afc70 con 0x5560e80e1150 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ad540 con 0x5560e80ea650 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f585c5f6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ab370 con 0x5560e80e1a70 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f58537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 0x5560e80e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48554/0 (socket says 172.21.15.49:48554) 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f5852ffd640 1 -- 172.21.15.49:0/867664903 learned_addr learned my addr 172.21.15.49:0/867664903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.282+0000 7f58527fc640 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 -- 172.21.15.49:0/867664903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 msgr2=0x5560e80e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 -- 172.21.15.49:0/867664903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 msgr2=0x5560e80e29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 0x5560e80e29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 -- 172.21.15.49:0/867664903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e7d190f0 con 0x5560e80e1150 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f58537fe640 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 0x5560e80e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f5852ffd640 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5844005820 tx=0x7f584404b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a3f83ceb9623f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f58397fa640 1 -- 172.21.15.49:0/867664903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584404b040 con 0x5560e80e1150 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f58397fa640 1 -- 172.21.15.49:0/867664903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58440bd500 con 0x5560e80e1150 2024-03-16T08:20:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.283+0000 7f58397fa640 1 -- 172.21.15.49:0/867664903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440bb100 con 0x5560e80e1150 2024-03-16T08:20:21.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- 172.21.15.49:0/867664903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 msgr2=0x5560e80e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5844005820 tx=0x7f584404b590 comp rx=0 tx=0).stop 2024-03-16T08:20:21.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- 172.21.15.49:0/867664903 shutdown_connections 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80ea650 0x5560e80e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1a70 0x5560e80e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 --2- 172.21.15.49:0/867664903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80e1150 0x5560e80e9da0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- 172.21.15.49:0/867664903 >> 172.21.15.49:0/867664903 conn(0x5560e80db880 msgr2=0x5560e80dd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- 172.21.15.49:0/867664903 shutdown_connections 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- 172.21.15.49:0/867664903 wait complete. 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 -- start start 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.284+0000 7f585c5f6740 1 Processor -- start 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58541aba00 con 0x5560e80ea650 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ace70 con 0x5560e80e1a70 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ac0d0 con 0x5560e80e1150 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f585c5f6740 10 client.0 _ll_register_callbacks cb 0x7f584c0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58527fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48558/0 (socket says 172.21.15.49:48558) 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58527fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:46016/0 (socket says 172.21.15.49:46016) 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 -- 172.21.15.49:0/2313756032 learned_addr learned my addr 172.21.15.49:0/2313756032 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 -- 172.21.15.49:0/2313756032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 msgr2=0x5560e80e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 -- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 msgr2=0x5560e80e48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58537fe640 1 -- 172.21.15.49:0/2313756032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e800c9d0 con 0x5560e80ea650 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f58527fc640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.285+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f58537fe640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f584c003e40 tx=0x7f584c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c48fd5b06c13928 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c036030 con 0x5560e80ea650 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f584c01a040 con 0x5560e80ea650 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58440bd500 con 0x5560e80ea650 2024-03-16T08:20:21.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f584c03f020 con 0x5560e80ea650 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 10 client.16467 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5560e8160c00 con 0x5560e80ea650 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58440bd500 con 0x5560e80ea650 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f584c027040 con 0x5560e80ea650 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 10 client.16467 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 20 client.16467 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e8161320 0x5560e8163740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 4 client.16467 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.286+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560e7fca540 con 0x5560e8161320 2024-03-16T08:20:21.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e8161320 0x5560e8163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:21.292 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:21 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5852ffd640 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e8161320 0x5560e8163740 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8a5f91ece9c565 server_cookie=b5abb1f05fb2488f in_seq=0 out_seq=0 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5832ffd640 10 client.16467 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5832ffd640 1 -- 172.21.15.49:0/2313756032 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5560e7fca540 con 0x5560e8161320 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5832ffd640 10 client.16467 handle_command_reply: tid=0 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 client.16467 shutdown 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 msgr2=0x5560e80e4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f584c003e40 tx=0x7f584c011ab0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 shutdown_connections 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5560e8161320 0x5560e8163740 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560e80ea650 0x5560e80e4e00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5560e80e1a70 0x5560e80e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f5832ffd640 0 client.16467 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 --2- 172.21.15.49:0/2313756032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e80e1150 0x5560e80e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.287+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 >> 172.21.15.49:0/2313756032 conn(0x5560e80db880 msgr2=0x5560e80da070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 shutdown_connections 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f585c5f6740 1 -- 172.21.15.49:0/2313756032 wait complete. 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f585c5f6740 20 client.16467 trim_cache size 0 max 16384 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout: "35eb933e-2349-4bdf-a78f-c3ed55f1edbe": { 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:20:21.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "35eb933e-2349-4bdf-a78f-c3ed55f1edbe", 2024-03-16T08:20:21.296 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:20:21.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:20:21.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:20:21.296 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 msgr2=0x7f583c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 0x7f583c0a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f584c005c40 tx=0x7f584c009c10 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 msgr2=0x7f58541980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5844009230 tx=0x7f584402cb30 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 shutdown_connections 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f583c0a2970 0x7f583c0a4e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5854106680 0x7f585419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5854101030 0x7f5854198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 --2- 172.21.15.49:0/2668817348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5854100670 0x7f58541980e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 >> 172.21.15.49:0/2668817348 conn(0x7f58540777b0 msgr2=0x7f585406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 shutdown_connections 2024-03-16T08:20:21.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:21.288+0000 7f5859937640 1 -- 172.21.15.49:0/2668817348 wait complete. 2024-03-16T08:20:21.299 INFO:tasks.cephfs.filesystem:scrub status for tag:35eb933e-2349-4bdf-a78f-c3ed55f1edbe - {'path': '/', 'tag': '35eb933e-2349-4bdf-a78f-c3ed55f1edbe', 'options': 'recursive,force'} 2024-03-16T08:20:21.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:21 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:20:21.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:21 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:20:22.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:22.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:20:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:22 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:20:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:20:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:22 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:20:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:20:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:22 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:20:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:24 smithi129 ceph-mon[36561]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:24 smithi154 ceph-mon[36471]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:24.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:24 smithi049 ceph-mon[31942]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:26.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:26 smithi129 ceph-mon[36561]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:26.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:26 smithi154 ceph-mon[36471]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:26 smithi049 ceph-mon[31942]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:28.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:28 smithi154 ceph-mon[36471]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:28.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:28 smithi129 ceph-mon[36561]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:28 smithi049 ceph-mon[31942]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:30.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:30 smithi129 ceph-mon[36561]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:30.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:30 smithi154 ceph-mon[36471]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:30.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:30 smithi049 ceph-mon[31942]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:32 smithi129 ceph-mon[36561]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:32.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:32 smithi154 ceph-mon[36471]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:32 smithi049 ceph-mon[31942]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:34 smithi129 ceph-mon[36561]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:34.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:34 smithi154 ceph-mon[36471]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:34.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:34 smithi049 ceph-mon[31942]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:36.340 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:36 smithi049 ceph-mon[31942]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:36 smithi129 ceph-mon[36561]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:36 smithi154 ceph-mon[36471]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:38.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:38 smithi129 ceph-mon[36561]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:38 smithi154 ceph-mon[36471]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:38.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:38 smithi049 ceph-mon[31942]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:40 smithi129 ceph-mon[36561]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:40 smithi154 ceph-mon[36471]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:40 smithi049 ceph-mon[31942]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:42 smithi129 ceph-mon[36561]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:42 smithi154 ceph-mon[36471]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:42.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:42 smithi049 ceph-mon[31942]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:44 smithi129 ceph-mon[36561]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:44.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:44 smithi154 ceph-mon[36471]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:44 smithi049 ceph-mon[31942]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:46 smithi129 ceph-mon[36561]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:46 smithi154 ceph-mon[36471]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:46 smithi049 ceph-mon[31942]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:48.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:48 smithi129 ceph-mon[36561]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:48 smithi154 ceph-mon[36471]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:48.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:48 smithi049 ceph-mon[31942]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:50 smithi129 ceph-mon[36561]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:50.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:50 smithi154 ceph-mon[36471]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:50 smithi049 ceph-mon[31942]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:20:51.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:20:51.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:20:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 -- 172.21.15.49:0/1580467214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 msgr2=0x7f0ab0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 --2- 172.21.15.49:0/1580467214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab0068dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009900 tx=0x7f0aac02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 -- 172.21.15.49:0/1580467214 shutdown_connections 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 --2- 172.21.15.49:0/1580467214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab010e180 0x7f0ab0110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 --2- 172.21.15.49:0/1580467214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab0069300 0x7f0ab010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 --2- 172.21.15.49:0/1580467214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab0068dc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 -- 172.21.15.49:0/1580467214 >> 172.21.15.49:0/1580467214 conn(0x7f0ab00777e0 msgr2=0x7f0ab0077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 -- 172.21.15.49:0/1580467214 shutdown_connections 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.425+0000 7f0ab7532640 1 -- 172.21.15.49:0/1580467214 wait complete. 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 Processor -- start 2024-03-16T08:20:51.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 -- start start 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab0069300 0x7f0ab019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab010e180 0x7f0ab019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0113d50 con 0x7f0ab00689e0 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0113bd0 con 0x7f0ab0069300 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab7532640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab0113ed0 con 0x7f0ab010e180 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59232/0 (socket says 172.21.15.49:59232) 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab52ce640 1 -- 172.21.15.49:0/4025848993 learned_addr learned my addr 172.21.15.49:0/4025848993 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab010e180 0x7f0ab019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.426+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab0069300 0x7f0ab019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 -- 172.21.15.49:0/4025848993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab010e180 msgr2=0x7f0ab019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab010e180 0x7f0ab019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 -- 172.21.15.49:0/4025848993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab0069300 msgr2=0x7f0ab019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab0069300 0x7f0ab019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 -- 172.21.15.49:0/4025848993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab019f260 con 0x7f0ab00689e0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006010 tx=0x7f0aac02cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0aa67fc640 1 -- 172.21.15.49:0/4025848993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac043020 con 0x7f0ab00689e0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0aa67fc640 1 -- 172.21.15.49:0/4025848993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aac0054e0 con 0x7f0ab00689e0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab019f4f0 con 0x7f0ab00689e0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0aa67fc640 1 -- 172.21.15.49:0/4025848993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003b90 con 0x7f0ab00689e0 2024-03-16T08:20:51.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.427+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab01a6de0 con 0x7f0ab00689e0 2024-03-16T08:20:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.429+0000 7f0aa67fc640 1 -- 172.21.15.49:0/4025848993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0aac046020 con 0x7f0ab00689e0 2024-03-16T08:20:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 Processor -- start 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 -- start start 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f5c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572c1ec0500 con 0x5572c1f65650 2024-03-16T08:20:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572c1e68be0 con 0x5572c1f5ca70 2024-03-16T08:20:51.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5572c1ebc5a0 con 0x5572c1f5c150 2024-03-16T08:20:51.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aa67fc640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 0x7f0a9c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59238/0 (socket says 172.21.15.49:59238) 2024-03-16T08:20:51.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 -- 172.21.15.49:0/2262808656 learned_addr learned my addr 172.21.15.49:0/2262808656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f5c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0aa67fc640 1 -- 172.21.15.49:0/4025848993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0aac0f0ce0 con 0x7f0ab00689e0 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 0x7f0a9c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 -- 172.21.15.49:0/2262808656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 msgr2=0x5572c1f5c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f5c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 -- 172.21.15.49:0/2262808656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 msgr2=0x5572c1f65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.430+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f65110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0ab4acd640 1 -- 172.21.15.49:0/2262808656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c1e7c4b0 con 0x5572c1f65650 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009c60 tx=0x7f0aa0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6644113e682696 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 0x7f0a9c0a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009b80 tx=0x7f0aa8009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0a9b7fe640 1 -- 172.21.15.49:0/2262808656 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0004220 con 0x5572c1f65650 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0a9b7fe640 1 -- 172.21.15.49:0/2262808656 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa00043c0 con 0x5572c1f65650 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.431+0000 7f0a9b7fe640 1 -- 172.21.15.49:0/2262808656 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0019910 con 0x5572c1f65650 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2262808656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 msgr2=0x5572c1f67a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009c60 tx=0x7f0aa0011810 comp rx=0 tx=0).stop 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2262808656 shutdown_connections 2024-03-16T08:20:51.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f67a40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2262808656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f5c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2262808656 >> 172.21.15.49:0/2262808656 conn(0x5572c1f56880 msgr2=0x5572c1f56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2262808656 shutdown_connections 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2262808656 wait complete. 2024-03-16T08:20:51.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 -- start start 2024-03-16T08:20:51.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.432+0000 7f0aba1f4740 1 Processor -- start 2024-03-16T08:20:51.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f62b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572c1f68440 con 0x5572c1f65650 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572c1e7c160 con 0x5572c1f5ca70 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5572c1f685c0 con 0x5572c1f5c150 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0aba1f4740 10 client.0 _ll_register_callbacks cb 0x5572c1e60150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59252/0 (socket says 172.21.15.49:59252) 2024-03-16T08:20:51.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 -- 172.21.15.49:0/2156321825 learned_addr learned my addr 172.21.15.49:0/2156321825 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f62b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 -- 172.21.15.49:0/2156321825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 msgr2=0x5572c1f62630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f62630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 -- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 msgr2=0x5572c1f62b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f62b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 -- 172.21.15.49:0/2156321825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c1c9e7d0 con 0x5572c1f65650 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f62b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8011620 tx=0x7f0aa8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94791fb2e63e2695 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa802f060 con 0x5572c1f65650 2024-03-16T08:20:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.433+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0aa800fe40 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aa00043c0 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0aa801fce0 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 10 client.16479 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5572c1bd43f0 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa00043c0 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0aa801d070 con 0x5572c1f65650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 10 client.16479 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 20 client.16479 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1f75650 0x5572c1f77aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 4 client.16479 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c1ecda40 con 0x5572c1f75650 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.434+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1f75650 0x5572c1f77aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.435+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1f75650 0x5572c1f77aa0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513d6332e5831fac server_cookie=49271481a4cb8391 in_seq=0 out_seq=0 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.435+0000 7f0a98ff9640 10 client.16479 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.435+0000 7f0a98ff9640 1 -- 172.21.15.49:0/2156321825 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572c1ecda40 con 0x5572c1f75650 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.435+0000 7f0a98ff9640 10 client.16479 handle_command_reply: tid=0 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.435+0000 7f0aba1f4740 1 client.16479 shutdown 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 msgr2=0x5572c1f630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8011620 tx=0x7f0aa8004ab0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 shutdown_connections 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1f75650 0x5572c1f77aa0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f630b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f62b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/2156321825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5c150 0x5572c1f62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0a98ff9640 0 client.16479 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 >> 172.21.15.49:0/2156321825 conn(0x5572c1f56880 msgr2=0x5572c1f585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 shutdown_connections 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 1 -- 172.21.15.49:0/2156321825 wait complete. 2024-03-16T08:20:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.436+0000 7f0aba1f4740 20 client.16479 trim_cache size 0 max 16384 2024-03-16T08:20:51.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.453+0000 7f0aba1f4740 1 Processor -- start 2024-03-16T08:20:51.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.453+0000 7f0aba1f4740 1 -- start start 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 0x5572c1f64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5ca70 0x5572c1f61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01b1e60 con 0x5572c1f65650 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01af730 con 0x5572c1f5c150 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01ad560 con 0x5572c1f5ca70 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab5acf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab5acf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59258/0 (socket says 172.21.15.49:59258) 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 0x5572c1f64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab5acf640 1 -- 172.21.15.49:0/147083492 learned_addr learned my addr 172.21.15.49:0/147083492 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5ca70 0x5572c1f61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 -- 172.21.15.49:0/147083492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5ca70 msgr2=0x5572c1f61570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5ca70 0x5572c1f61570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 -- 172.21.15.49:0/147083492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 msgr2=0x5572c1f711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 -- 172.21.15.49:0/147083492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c1b940f0 con 0x5572c1f5c150 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 0x5572c1f64d60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ee590 tx=0x7f0aac0dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af786ad2577b5987 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.454+0000 7f0a9affd640 1 -- 172.21.15.49:0/147083492 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0bdba0 con 0x5572c1f5c150 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0a9affd640 1 -- 172.21.15.49:0/147083492 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aac0bd4a0 con 0x5572c1f5c150 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0a9affd640 1 -- 172.21.15.49:0/147083492 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003e50 con 0x5572c1f5c150 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 -- 172.21.15.49:0/147083492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 msgr2=0x5572c1f64d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 0x5572c1f64d60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ee590 tx=0x7f0aac0dd040 comp rx=0 tx=0).stop 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 -- 172.21.15.49:0/147083492 shutdown_connections 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f65650 0x5572c1f711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f5ca70 0x5572c1f61570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/147083492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5c150 0x5572c1f64d60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 -- 172.21.15.49:0/147083492 >> 172.21.15.49:0/147083492 conn(0x5572c1f56880 msgr2=0x5572c1f62e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 -- 172.21.15.49:0/147083492 shutdown_connections 2024-03-16T08:20:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.455+0000 7f0aba1f4740 1 -- 172.21.15.49:0/147083492 wait complete. 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 -- start start 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 Processor -- start 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f5f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f65650 0x5572c1f5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01adbf0 con 0x5572c1f5c150 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01af060 con 0x5572c1f5ca70 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab01ae2c0 con 0x5572c1f65650 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0aba1f4740 10 client.0 _ll_register_callbacks cb 0x7f0aa8009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59262/0 (socket says 172.21.15.49:59262) 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 -- 172.21.15.49:0/3450201502 learned_addr learned my addr 172.21.15.49:0/3450201502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f5f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab5acf640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f65650 0x5572c1f5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 -- 172.21.15.49:0/3450201502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f65650 msgr2=0x5572c1f5fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f65650 0x5572c1f5fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 -- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 msgr2=0x5572c1f5f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.456+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f5f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0ab52ce640 1 -- 172.21.15.49:0/3450201502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c1e879d0 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0ab52ce640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0c5710 tx=0x7f0aac02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41025a52ff20cc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003e30 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0aac0042e0 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac0bd4a0 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0aac0bdba0 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 10 client.16485 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5572c1fdbb40 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aac0bd4a0 con 0x5572c1f5c150 2024-03-16T08:20:51.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0aac0c1170 con 0x5572c1f5c150 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 10 client.16485 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 20 client.16485 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1fdc240 0x5572c1fde660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 4 client.16485 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.457+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572c1ec7d90 con 0x5572c1fdc240 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1fdc240 0x5572c1fde660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0ab4acd640 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1fdc240 0x5572c1fde660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e434d4a9003d481 server_cookie=b52f857cb10e542d in_seq=0 out_seq=0 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0a98ff9640 10 client.16485 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0a98ff9640 1 -- 172.21.15.49:0/3450201502 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5572c1ec7d90 con 0x5572c1fdc240 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0a98ff9640 10 client.16485 handle_command_reply: tid=0 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 client.16485 shutdown 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 msgr2=0x5572c1f5f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0c5710 tx=0x7f0aac02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 shutdown_connections 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5572c1fdc240 0x5572c1fde660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572c1f65650 0x5572c1f5fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572c1f5ca70 0x5572c1f5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0a98ff9640 0 client.16485 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 --2- 172.21.15.49:0/3450201502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1f5c150 0x5572c1f5f260 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 >> 172.21.15.49:0/3450201502 conn(0x5572c1f56880 msgr2=0x5572c1f55070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 shutdown_connections 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.458+0000 7f0aba1f4740 1 -- 172.21.15.49:0/3450201502 wait complete. 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0aba1f4740 20 client.16485 trim_cache size 0 max 16384 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:51.467 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:20:51.468 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:20:51.468 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:51.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 msgr2=0x7f0a9c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 0x7f0a9c0a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009b80 tx=0x7f0aa8009500 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 msgr2=0x7f0ab019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006010 tx=0x7f0aac02cbd0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 shutdown_connections 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0a9c0a2970 0x7f0a9c0a4e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab010e180 0x7f0ab019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ab0069300 0x7f0ab019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 --2- 172.21.15.49:0/4025848993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab00689e0 0x7f0ab019a270 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 >> 172.21.15.49:0/4025848993 conn(0x7f0ab00777e0 msgr2=0x7f0ab010f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 shutdown_connections 2024-03-16T08:20:51.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.459+0000 7f0ab7532640 1 -- 172.21.15.49:0/4025848993 wait complete. 2024-03-16T08:20:51.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:20:51.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:20:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 -- 172.21.15.49:0/1043600772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 msgr2=0x7f77fc10e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 --2- 172.21.15.49:0/1043600772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc10e630 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009ca0 tx=0x7f77f402c180 comp rx=0 tx=0).stop 2024-03-16T08:20:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 -- 172.21.15.49:0/1043600772 shutdown_connections 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 --2- 172.21.15.49:0/1043600772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc10e630 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 --2- 172.21.15.49:0/1043600772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 0x7f77fc06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 --2- 172.21.15.49:0/1043600772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 0x7f77fc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 -- 172.21.15.49:0/1043600772 >> 172.21.15.49:0/1043600772 conn(0x7f77fc077780 msgr2=0x7f77fc077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 -- 172.21.15.49:0/1043600772 shutdown_connections 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.579+0000 7f7803efe640 1 -- 172.21.15.49:0/1043600772 wait complete. 2024-03-16T08:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 Processor -- start 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 -- start start 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 0x7f77fc100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 0x7f77fc101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111ba0 con 0x7f77fc06a830 2024-03-16T08:20:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111a20 con 0x7f77fc1077e0 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7803efe640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111d20 con 0x7f77fc069e90 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7801499640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7801499640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59268/0 (socket says 172.21.15.49:59268) 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7801499640 1 -- 172.21.15.49:0/2723306737 learned_addr learned my addr 172.21.15.49:0/2723306737 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f7801c9a640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 0x7f77fc100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.580+0000 7f780249b640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 0x7f77fc101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 -- 172.21.15.49:0/2723306737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 msgr2=0x7f77fc100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 0x7f77fc100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 -- 172.21.15.49:0/2723306737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 msgr2=0x7f77fc101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 0x7f77fc101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 -- 172.21.15.49:0/2723306737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1a64c0 con 0x7f77fc06a830 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7801499640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e9f0 tx=0x7f77ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec00cd80 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec005500 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1a67b0 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec010690 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.581+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a6cb0 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.582+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc102f00 con 0x7f77fc06a830 2024-03-16T08:20:51.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.583+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f77ec0046c0 con 0x7f77fc06a830 2024-03-16T08:20:51.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.584+0000 7f77f2ffd640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 0x7f77e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.584+0000 7f7801c9a640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 0x7f77e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.584+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f77ec0cf3f0 con 0x7f77fc06a830 2024-03-16T08:20:51.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.585+0000 7f7801c9a640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 0x7f77e80a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009b50 tx=0x7f77f8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.588+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77ec053740 con 0x7f77fc06a830 2024-03-16T08:20:51.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.809+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77fc068a20 con 0x7f77fc06a830 2024-03-16T08:20:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.810+0000 7f77f2ffd640 1 -- 172.21.15.49:0/2723306737 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f77ec09c1a0 con 0x7f77fc06a830 2024-03-16T08:20:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:20:51.812 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:20:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:20:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 msgr2=0x7f77e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 0x7f77e80a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009b50 tx=0x7f77f8005f90 comp rx=0 tx=0).stop 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 msgr2=0x7f77fc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e9f0 tx=0x7f77ec00edf0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 shutdown_connections 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77e80a2970 0x7f77e80a4e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77fc1077e0 0x7f77fc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77fc06a830 0x7f77fc100c50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 --2- 172.21.15.49:0/2723306737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77fc069e90 0x7f77fc100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 >> 172.21.15.49:0/2723306737 conn(0x7f77fc077780 msgr2=0x7f77fc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 shutdown_connections 2024-03-16T08:20:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.812+0000 7f7803efe640 1 -- 172.21.15.49:0/2723306737 wait complete. 2024-03-16T08:20:51.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:20:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 -- 172.21.15.49:0/3238075131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8101780 msgr2=0x7f5ce8101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 --2- 172.21.15.49:0/3238075131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8101780 0x7f5ce8101c00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009930 tx=0x7f5cdc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:20:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 -- 172.21.15.49:0/3238075131 shutdown_connections 2024-03-16T08:20:51.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 --2- 172.21.15.49:0/3238075131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8102140 0x7f5ce810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 --2- 172.21.15.49:0/3238075131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8101780 0x7f5ce8101c00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 --2- 172.21.15.49:0/3238075131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8107790 0x7f5ce8107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 -- 172.21.15.49:0/3238075131 >> 172.21.15.49:0/3238075131 conn(0x7f5ce80fd210 msgr2=0x7f5ce80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 -- 172.21.15.49:0/3238075131 shutdown_connections 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.935+0000 7f5cedddf640 1 -- 172.21.15.49:0/3238075131 wait complete. 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 Processor -- start 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 -- start start 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8101780 0x7f5ce8073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8107790 0x7f5ce8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8111bf0 con 0x7f5ce8102140 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8111a70 con 0x7f5ce8107790 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5cedddf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8111d70 con 0x7f5ce8101780 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59280/0 (socket says 172.21.15.49:59280) 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/1205370272 learned_addr learned my addr 172.21.15.49:0/1205370272 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8107790 0x7f5ce8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.936+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8101780 0x7f5ce8073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/1205370272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8101780 msgr2=0x7f5ce8073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8101780 0x7f5ce8073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/1205370272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8107790 msgr2=0x7f5ce8074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8107790 0x7f5ce8074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/1205370272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce81a65b0 con 0x7f5ce8102140 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009900 tx=0x7f5cdc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce4ff9640 1 -- 172.21.15.49:0/1205370272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc043020 con 0x7f5ce8102140 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce81a6840 con 0x7f5ce8102140 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce4ff9640 1 -- 172.21.15.49:0/1205370272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc0054e0 con 0x7f5ce8102140 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5ce4ff9640 1 -- 172.21.15.49:0/1205370272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc003bb0 con 0x7f5ce8102140 2024-03-16T08:20:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.937+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce81a6de0 con 0x7f5ce8102140 2024-03-16T08:20:51.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 Processor -- start 2024-03-16T08:20:51.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 -- start start 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5ce4ff9640 1 -- 172.21.15.49:0/1205370272 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5cdc046020 con 0x7f5ce8102140 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5c92af490 con 0x55f5c9354d30 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9257c90 con 0x55f5c934c150 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.940+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f5c92ab530 con 0x55f5c934b830 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37906/0 (socket says 172.21.15.49:37906) 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42434/0 (socket says 172.21.15.49:42434) 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59290/0 (socket says 172.21.15.49:59290) 2024-03-16T08:20:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 -- 172.21.15.49:0/3428752451 learned_addr learned my addr 172.21.15.49:0/3428752451 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 -- 172.21.15.49:0/3428752451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 msgr2=0x55f5c93547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 -- 172.21.15.49:0/3428752451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 msgr2=0x55f5c9357120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 -- 172.21.15.49:0/3428752451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c9312500 con 0x55f5c934b830 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce4ff9640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 0x7f5cd00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 0x7f5cd00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce4ff9640 1 -- 172.21.15.49:0/1205370272 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5cdc0f0ce0 con 0x7f5ce8102140 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009fd0 tx=0x7f5ce00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678e2a222ea9464b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ccdffb640 1 -- 172.21.15.49:0/3428752451 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0004d20 con 0x55f5c934b830 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ccdffb640 1 -- 172.21.15.49:0/3428752451 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce0004050 con 0x55f5c934b830 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.941+0000 7f5ccdffb640 1 -- 172.21.15.49:0/3428752451 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0005240 con 0x55f5c934b830 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- 172.21.15.49:0/3428752451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 msgr2=0x55f5c934bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009fd0 tx=0x7f5ce00118d0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 0x7f5cd00a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55f5c934b300 tx=0x7f5cd8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- 172.21.15.49:0/3428752451 shutdown_connections 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9357120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934c150 0x55f5c93547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/3428752451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c934bc10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- 172.21.15.49:0/3428752451 >> 172.21.15.49:0/3428752451 conn(0x55f5c9345f60 msgr2=0x55f5c9346340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- 172.21.15.49:0/3428752451 shutdown_connections 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- 172.21.15.49:0/3428752451 wait complete. 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.942+0000 7f5cf09c9740 1 -- start start 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 Processor -- start 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c9351d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c9354d30 0x55f5c9352780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9357b20 con 0x55f5c934c150 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f5c926b230 con 0x55f5c9354d30 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f5c9357ca0 con 0x55f5c934b830 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5cf09c9740 10 client.0 _ll_register_callbacks cb 0x55f5c92a7010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59300/0 (socket says 172.21.15.49:59300) 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c9351d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/329008105 learned_addr learned my addr 172.21.15.49:0/329008105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c9354d30 0x55f5c9352780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 -- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c9354d30 msgr2=0x55f5c9352780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c9354d30 0x55f5c9352780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 -- 172.21.15.49:0/329008105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 msgr2=0x55f5c9352240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce77fe640 1 -- 172.21.15.49:0/329008105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c927a100 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.943+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c9351d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80045f0 tx=0x7f5cd8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7259489abdf80b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8005020 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5cd80051c0 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce0004050 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5cd80053c0 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 10 client.26386 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f5c8fc33f0 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce0004050 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5cd8027070 con 0x55f5c934b830 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 10 client.26386 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 10 client.26386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 20 client.26386 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c9365100 0x55f5c9367550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 4 client.26386 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.944+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c92bc9d0 con 0x55f5c9365100 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.945+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c9365100 0x55f5c9367550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.945+0000 7f5cc6ffd640 10 client.26386 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.945+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c9365100 0x55f5c9367550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bb3c17d5dfb819f server_cookie=f698cd3af9345323 in_seq=0 out_seq=0 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/329008105 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f5c92bc9d0 con 0x55f5c9365100 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cc6ffd640 10 client.26386 handle_command_reply: tid=0 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 client.26386 shutdown 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 msgr2=0x55f5c9351d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c9351d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80045f0 tx=0x7f5cd8004730 comp rx=0 tx=0).stop 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 shutdown_connections 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c9365100 0x55f5c9367550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c9354d30 0x55f5c9352780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cc6ffd640 0 client.26386 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c9352240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/329008105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934b830 0x55f5c9351d00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 >> 172.21.15.49:0/329008105 conn(0x55f5c9345f60 msgr2=0x55f5c9347cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 shutdown_connections 2024-03-16T08:20:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 1 -- 172.21.15.49:0/329008105 wait complete. 2024-03-16T08:20:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.946+0000 7f5cf09c9740 20 client.26386 trim_cache size 0 max 16384 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 Processor -- start 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 -- start start 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c9354430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 0x55f5c936ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81b1f60 con 0x55f5c934c150 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81af830 con 0x55f5c934b830 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81ad660 con 0x55f5c9354d30 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59308/0 (socket says 172.21.15.49:59308) 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/842152393 learned_addr learned my addr 172.21.15.49:0/842152393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c9354430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 0x55f5c936ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/842152393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 msgr2=0x55f5c936ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 0x55f5c936ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/842152393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 msgr2=0x55f5c9354430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c9354430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 -- 172.21.15.49:0/842152393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c908d7d0 con 0x55f5c934c150 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 0x55f5c936ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.960+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0b9e70 tx=0x7f5cdc0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad3d45066e15069 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5ccd7fa640 1 -- 172.21.15.49:0/842152393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0b9740 con 0x55f5c934c150 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5ccd7fa640 1 -- 172.21.15.49:0/842152393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc004c80 con 0x55f5c934c150 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5ccd7fa640 1 -- 172.21.15.49:0/842152393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0bce00 con 0x55f5c934c150 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 -- 172.21.15.49:0/842152393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 msgr2=0x55f5c93608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0b9e70 tx=0x7f5cdc0dd040 comp rx=0 tx=0).stop 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 -- 172.21.15.49:0/842152393 shutdown_connections 2024-03-16T08:20:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c9354d30 0x55f5c936ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c934c150 0x55f5c93608c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/842152393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c9354430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 -- 172.21.15.49:0/842152393 >> 172.21.15.49:0/842152393 conn(0x55f5c9345f60 msgr2=0x55f5c9344750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 -- 172.21.15.49:0/842152393 shutdown_connections 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.961+0000 7f5cf09c9740 1 -- 172.21.15.49:0/842152393 wait complete. 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 -- start start 2024-03-16T08:20:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 Processor -- start 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c93587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934c150 0x55f5c9358d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81adcf0 con 0x55f5c9354d30 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81af160 con 0x55f5c934b830 2024-03-16T08:20:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81ae3c0 con 0x55f5c934c150 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5cf09c9740 10 client.0 _ll_register_callbacks cb 0x7f5cd8009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59322/0 (socket says 172.21.15.49:59322) 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 -- 172.21.15.49:0/2015251702 learned_addr learned my addr 172.21.15.49:0/2015251702 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce6ffd640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934c150 0x55f5c9358d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c93587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 -- 172.21.15.49:0/2015251702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934c150 msgr2=0x55f5c9358d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934c150 0x55f5c9358d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 -- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 msgr2=0x55f5c93587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c93587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.962+0000 7f5ce7fff640 1 -- 172.21.15.49:0/2015251702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c93ca450 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5ce7fff640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009f70 tx=0x7f5ce00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87e7856ec270ec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0003b80 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5ce0003d20 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc004c80 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0004000 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 10 client.16509 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f5c93cb720 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc004c80 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ce0005490 con 0x55f5c9354d30 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 10 client.16509 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 10 client.16509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:20:51.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 20 client.16509 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c93cbc40 0x55f5c93ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 4 client.16509 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.963+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5c92b6d20 con 0x55f5c93cbc40 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c93cbc40 0x55f5c93ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5ce77fe640 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c93cbc40 0x55f5c93ce030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f703fef372a21c66 server_cookie=14dd4712435559e7 in_seq=0 out_seq=0 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cc6ffd640 10 client.16509 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cc6ffd640 1 -- 172.21.15.49:0/2015251702 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5c92b6d20 con 0x55f5c93cbc40 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cc6ffd640 10 client.16509 handle_command_reply: tid=0 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 client.16509 shutdown 2024-03-16T08:20:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 msgr2=0x55f5c9359240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009f70 tx=0x7f5ce00095b0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 shutdown_connections 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55f5c93cbc40 0x55f5c93ce030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5c9354d30 0x55f5c9359240 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cc6ffd640 0 client.16509 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f5c934c150 0x55f5c9358d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 --2- 172.21.15.49:0/2015251702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f5c934b830 0x55f5c93587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 >> 172.21.15.49:0/2015251702 conn(0x55f5c9345f60 msgr2=0x55f5c9347af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 shutdown_connections 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 1 -- 172.21.15.49:0/2015251702 wait complete. 2024-03-16T08:20:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.964+0000 7f5cf09c9740 20 client.16509 trim_cache size 0 max 16384 2024-03-16T08:20:51.974 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 msgr2=0x7f5cd00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 0x7f5cd00a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55f5c934b300 tx=0x7f5cd8005f90 comp rx=0 tx=0).stop 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 msgr2=0x7f5ce8074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009900 tx=0x7f5cdc004360 comp rx=0 tx=0).stop 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 shutdown_connections 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5cd00a2970 0x7f5cd00a4e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ce8107790 0x7f5ce8074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ce8102140 0x7f5ce8074020 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 --2- 172.21.15.49:0/1205370272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ce8101780 0x7f5ce8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 >> 172.21.15.49:0/1205370272 conn(0x7f5ce80fd210 msgr2=0x7f5ce80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 shutdown_connections 2024-03-16T08:20:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:51.965+0000 7f5cedddf640 1 -- 172.21.15.49:0/1205370272 wait complete. 2024-03-16T08:20:51.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 -- 172.21.15.49:0/3878037342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f300689c0 msgr2=0x7f3f30068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/3878037342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f300689c0 0x7f3f30068da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009860 tx=0x7f3f2002bb70 comp rx=0 tx=0).stop 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 -- 172.21.15.49:0/3878037342 shutdown_connections 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/3878037342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f3010e180 0x7f3f30110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/3878037342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 0x7f3f3010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/3878037342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f300689c0 0x7f3f30068da0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 -- 172.21.15.49:0/3878037342 >> 172.21.15.49:0/3878037342 conn(0x7f3f300777e0 msgr2=0x7f3f30077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 -- 172.21.15.49:0/3878037342 shutdown_connections 2024-03-16T08:20:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.090+0000 7f3f35fb3640 1 -- 172.21.15.49:0/3878037342 wait complete. 2024-03-16T08:20:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 Processor -- start 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 -- start start 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 0x7f3f3019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f3010e180 0x7f3f3019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113d40 con 0x7f3f300689c0 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113bc0 con 0x7f3f300692e0 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f35fb3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113ec0 con 0x7f3f3010e180 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59338/0 (socket says 172.21.15.49:59338) 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f2f7fe640 1 -- 172.21.15.49:0/2714116479 learned_addr learned my addr 172.21.15.49:0/2714116479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.091+0000 7f3f2effd640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 0x7f3f3019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f3010e180 0x7f3f3019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 -- 172.21.15.49:0/2714116479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f3010e180 msgr2=0x7f3f3019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f3010e180 0x7f3f3019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 -- 172.21.15.49:0/2714116479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 msgr2=0x7f3f3019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 0x7f3f3019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 -- 172.21.15.49:0/2714116479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3019f270 con 0x7f3f300689c0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2f7fe640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3f20037040 tx=0x7f3f200045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2cff9640 1 -- 172.21.15.49:0/2714116479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20041020 con 0x7f3f300689c0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3019f500 con 0x7f3f300689c0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2cff9640 1 -- 172.21.15.49:0/2714116479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f2002c9f0 con 0x7f3f300689c0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f2cff9640 1 -- 172.21.15.49:0/2714116479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20039040 con 0x7f3f300689c0 2024-03-16T08:20:52.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.092+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a6df0 con 0x7f3f300689c0 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.094+0000 7f3f2cff9640 1 -- 172.21.15.49:0/2714116479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3f20044020 con 0x7f3f300689c0 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.094+0000 7f3f38b9a740 1 Processor -- start 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 -- start start 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 0x55aa73ee6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2cff9640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 0x7f3f180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73eefd30 0x55aa73ef2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aa73e4a490 con 0x55aa73ee7150 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa73df2c90 con 0x55aa73ee6830 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55aa73e46530 con 0x55aa73eefd30 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2f7fe640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 0x7f3f180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59348/0 (socket says 172.21.15.49:59348) 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 -- 172.21.15.49:0/1676735662 learned_addr learned my addr 172.21.15.49:0/1676735662 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 0x55aa73ee6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2cff9640 1 -- 172.21.15.49:0/2714116479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3f200f0530 con 0x7f3f300689c0 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73eefd30 0x55aa73ef2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 -- 172.21.15.49:0/1676735662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73eefd30 msgr2=0x55aa73ef2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73eefd30 0x55aa73ef2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 -- 172.21.15.49:0/1676735662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 msgr2=0x55aa73ee6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 0x55aa73ee6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2effd640 1 -- 172.21.15.49:0/1676735662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa73ead500 con 0x55aa73ee7150 2024-03-16T08:20:52.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 0x55aa73ee6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.095+0000 7f3f2f7fe640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 0x7f3f180a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f200bc9f0 tx=0x7f3f200b2180 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009ca0 tx=0x7f3f24011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e7731edadc782 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f15ffb640 1 -- 172.21.15.49:0/1676735662 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24004230 con 0x55aa73ee7150 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f15ffb640 1 -- 172.21.15.49:0/1676735662 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f240043d0 con 0x55aa73ee7150 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f15ffb640 1 -- 172.21.15.49:0/1676735662 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240035f0 con 0x55aa73ee7150 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1676735662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 msgr2=0x55aa73eef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009ca0 tx=0x7f3f24011810 comp rx=0 tx=0).stop 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1676735662 shutdown_connections 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73eefd30 0x55aa73ef2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73eef7f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1676735662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee6830 0x55aa73ee6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1676735662 >> 172.21.15.49:0/1676735662 conn(0x55aa73ee0f60 msgr2=0x55aa73ee1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1676735662 shutdown_connections 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.096+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1676735662 wait complete. 2024-03-16T08:20:52.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 -- start start 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 Processor -- start 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 0x55aa73eed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73eefd30 0x55aa73eed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aa73ef2b20 con 0x55aa73eefd30 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa73e06230 con 0x55aa73ee7150 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55aa73ef2ca0 con 0x55aa73ee6830 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f38b9a740 10 client.0 _ll_register_callbacks cb 0x55aa73e42010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73eefd30 0x55aa73eed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 0x55aa73eed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37964/0 (socket says 172.21.15.49:37964) 2024-03-16T08:20:52.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.097+0000 7f3f2ffff640 1 -- 172.21.15.49:0/1262248803 learned_addr learned my addr 172.21.15.49:0/1262248803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 -- 172.21.15.49:0/1262248803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 msgr2=0x55aa73eed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 0x55aa73eed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 -- 172.21.15.49:0/1262248803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73eefd30 msgr2=0x55aa73eed810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73eefd30 0x55aa73eed810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 -- 172.21.15.49:0/1262248803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa73e15100 con 0x55aa73ee6830 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 0x55aa73eed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800f3f0 tx=0x7f3f28004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd33cc66b2adfd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28004b20 con 0x55aa73ee6830 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3f28004cc0 con 0x55aa73ee6830 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f240043d0 con 0x55aa73ee6830 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f28005020 con 0x55aa73ee6830 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f38b9a740 10 client.26404 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:52.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.098+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aa73b5e3f0 con 0x55aa73ee6830 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f240043d0 con 0x55aa73ee6830 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3f2800e5c0 con 0x55aa73ee6830 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 10 client.26404 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 10 client.26404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 20 client.26404 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f000e0 0x55aa73f02510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 4 client.26404 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa73e579d0 con 0x55aa73f000e0 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f000e0 0x55aa73f02510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f0effd640 10 client.26404 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.099+0000 7f3f2effd640 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f000e0 0x55aa73f02510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa54bbceb89d09c0 server_cookie=93bd6819a10786f5 in_seq=0 out_seq=0 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.100+0000 7f3f0effd640 1 -- 172.21.15.49:0/1262248803 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aa73e579d0 con 0x55aa73f000e0 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.100+0000 7f3f0effd640 10 client.26404 handle_command_reply: tid=0 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.100+0000 7f3f38b9a740 1 client.26404 shutdown 2024-03-16T08:20:52.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.100+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 msgr2=0x55aa73eecd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.100+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800f3f0 tx=0x7f3f28004ab0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 shutdown_connections 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f000e0 0x55aa73f02510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73eefd30 0x55aa73eed810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73ee7150 0x55aa73eed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/1262248803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eecd90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f0effd640 0 client.26404 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 >> 172.21.15.49:0/1262248803 conn(0x55aa73ee0f60 msgr2=0x55aa73ee2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 shutdown_connections 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 1 -- 172.21.15.49:0/1262248803 wait complete. 2024-03-16T08:20:52.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.101+0000 7f3f38b9a740 20 client.26404 trim_cache size 0 max 16384 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 Processor -- start 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 -- start start 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73f07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301b1e80 con 0x55aa73ee7150 2024-03-16T08:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301af750 con 0x55aa73eefd30 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ad580 con 0x55aa73ee6830 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59364/0 (socket says 172.21.15.49:59364) 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 -- 172.21.15.49:0/4134028017 learned_addr learned my addr 172.21.15.49:0/4134028017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73f07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 -- 172.21.15.49:0/4134028017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 msgr2=0x55aa73eef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 -- 172.21.15.49:0/4134028017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 msgr2=0x55aa73f07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73f07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.114+0000 7f3f2effd640 1 -- 172.21.15.49:0/4134028017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa73c287d0 con 0x55aa73ee7150 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f2effd640 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f0a3ee7b74d3c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f157fa640 1 -- 172.21.15.49:0/4134028017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24032070 con 0x55aa73ee7150 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f157fa640 1 -- 172.21.15.49:0/4134028017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f2401f040 con 0x55aa73ee7150 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f157fa640 1 -- 172.21.15.49:0/4134028017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2401e040 con 0x55aa73ee7150 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 -- 172.21.15.49:0/4134028017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 msgr2=0x55aa73efb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24004940 comp rx=0 tx=0).stop 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 -- 172.21.15.49:0/4134028017 shutdown_connections 2024-03-16T08:20:52.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73f07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73efb8c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/4134028017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73eef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 -- 172.21.15.49:0/4134028017 >> 172.21.15.49:0/4134028017 conn(0x55aa73ee0f60 msgr2=0x55aa73ee2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 -- 172.21.15.49:0/4134028017 shutdown_connections 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.115+0000 7f3f38b9a740 1 -- 172.21.15.49:0/4134028017 wait complete. 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 -- start start 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 Processor -- start 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73ef8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73ee9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301adc10 con 0x55aa73ee7150 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301af080 con 0x55aa73eefd30 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ae2e0 con 0x55aa73ee6830 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f38b9a740 10 client.0 _ll_register_callbacks cb 0x7f3f28012e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f2effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59366/0 (socket says 172.21.15.49:59366) 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f2effd640 1 -- 172.21.15.49:0/3172525573 learned_addr learned my addr 172.21.15.49:0/3172525573 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73ef8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.116+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73ee9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 -- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 msgr2=0x55aa73ef8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73ef8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 -- 172.21.15.49:0/3172525573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 msgr2=0x55aa73ee9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73ee9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 -- 172.21.15.49:0/3172525573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa73eea340 con 0x55aa73ee7150 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73ee9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f2effd640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006010 tx=0x7f3f24005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801cd478c9116349 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24032050 con 0x55aa73ee7150 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f3f24004740 con 0x55aa73ee7150 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f2401f040 con 0x55aa73ee7150 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f24019700 con 0x55aa73ee7150 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 10 client.16527 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:52.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aa73f66390 con 0x55aa73ee7150 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f2401f040 con 0x55aa73ee7150 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3f2402d070 con 0x55aa73ee7150 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 10 client.16527 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 10 client.16527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 20 client.16527 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f66920 0x55aa73f68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 4 client.16527 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.117+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa73e51d20 con 0x55aa73f66920 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f66920 0x55aa73f68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f2ffff640 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f66920 0x55aa73f68d40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0d3bf66582a3e3c server_cookie=f9ddcaefb8f94bd0 in_seq=0 out_seq=0 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f0effd640 10 client.16527 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f0effd640 1 -- 172.21.15.49:0/3172525573 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa73e51d20 con 0x55aa73f66920 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f0effd640 10 client.16527 handle_command_reply: tid=0 2024-03-16T08:20:52.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 client.16527 shutdown 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 msgr2=0x55aa73ee9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006010 tx=0x7f3f24005df0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 shutdown_connections 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55aa73f66920 0x55aa73f68d40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa73eefd30 0x55aa73ee9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aa73ee7150 0x55aa73ee9600 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f0effd640 0 client.16527 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 --2- 172.21.15.49:0/3172525573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa73ee6830 0x55aa73ef8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 >> 172.21.15.49:0/3172525573 conn(0x55aa73ee0f60 msgr2=0x55aa73eecb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 shutdown_connections 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.118+0000 7f3f38b9a740 1 -- 172.21.15.49:0/3172525573 wait complete. 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f38b9a740 20 client.16527 trim_cache size 0 max 16384 2024-03-16T08:20:52.129 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:52.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 msgr2=0x7f3f180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 0x7f3f180a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f200bc9f0 tx=0x7f3f200b2180 comp rx=0 tx=0).stop 2024-03-16T08:20:52.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 msgr2=0x7f3f3019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3f20037040 tx=0x7f3f200045b0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 shutdown_connections 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3f180a2970 0x7f3f180a4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f3010e180 0x7f3f3019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f300692e0 0x7f3f3019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 --2- 172.21.15.49:0/2714116479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f300689c0 0x7f3f3019a280 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 >> 172.21.15.49:0/2714116479 conn(0x7f3f300777e0 msgr2=0x7f3f3010f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 shutdown_connections 2024-03-16T08:20:52.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.119+0000 7f3f35fb3640 1 -- 172.21.15.49:0/2714116479 wait complete. 2024-03-16T08:20:52.132 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:20:52.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 -- 172.21.15.49:0/1592886966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0107790 msgr2=0x7f6bb0107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 --2- 172.21.15.49:0/1592886966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0107790 0x7f6bb0107b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009890 tx=0x7f6ba002bb70 comp rx=0 tx=0).stop 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 -- 172.21.15.49:0/1592886966 shutdown_connections 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 --2- 172.21.15.49:0/1592886966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0102140 0x7f6bb010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 --2- 172.21.15.49:0/1592886966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 --2- 172.21.15.49:0/1592886966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0107790 0x7f6bb0107b70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 -- 172.21.15.49:0/1592886966 >> 172.21.15.49:0/1592886966 conn(0x7f6bb00fd210 msgr2=0x7f6bb00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 -- 172.21.15.49:0/1592886966 shutdown_connections 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 -- 172.21.15.49:0/1592886966 wait complete. 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.241+0000 7f6bb6113640 1 Processor -- start 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 -- start start 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0102140 0x7f6bb0074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0107790 0x7f6bb0074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0111ba0 con 0x7f6bb0101780 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0111a20 con 0x7f6bb0107790 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6bb6113640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0111d20 con 0x7f6bb0102140 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59390/0 (socket says 172.21.15.49:59390) 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 -- 172.21.15.49:0/782223153 learned_addr learned my addr 172.21.15.49:0/782223153 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baffff640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0107790 0x7f6bb0074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baeffd640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0102140 0x7f6bb0074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 -- 172.21.15.49:0/782223153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0102140 msgr2=0x7f6bb0074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0102140 0x7f6bb0074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 -- 172.21.15.49:0/782223153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0107790 msgr2=0x7f6bb0074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0107790 0x7f6bb0074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 -- 172.21.15.49:0/782223153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb01a6640 con 0x7f6bb0101780 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.242+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6ba002c180 tx=0x7f6ba0004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.243+0000 7f6bacff9640 1 -- 172.21.15.49:0/782223153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0041020 con 0x7f6bb0101780 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.243+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb01a68d0 con 0x7f6bb0101780 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.243+0000 7f6bacff9640 1 -- 172.21.15.49:0/782223153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba002c9f0 con 0x7f6bb0101780 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.243+0000 7f6bacff9640 1 -- 172.21.15.49:0/782223153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0039040 con 0x7f6bb0101780 2024-03-16T08:20:52.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.243+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb01a6dc0 con 0x7f6bb0101780 2024-03-16T08:20:52.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 Processor -- start 2024-03-16T08:20:52.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bacff9640 1 -- 172.21.15.49:0/782223153 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6ba0044020 con 0x7f6bb0101780 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 -- start start 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 0x558320de77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320dea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558320d42490 con 0x558320dde830 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558320ceac90 con 0x558320ddf150 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558320d3e530 con 0x558320de7d30 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59402/0 (socket says 172.21.15.49:59402) 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6baffff640 1 -- 172.21.15.49:0/2745934054 learned_addr learned my addr 172.21.15.49:0/2745934054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 0x558320de77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.245+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320dea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6bacff9640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 0x7f6b980a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2745934054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 msgr2=0x558320dea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320dea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2745934054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 msgr2=0x558320ddec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2745934054 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558320da5500 con 0x558320ddf150 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320dea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baffff640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baffff640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 0x7f6b980a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6bacff9640 1 -- 172.21.15.49:0/782223153 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6ba00f0550 con 0x7f6bb0101780 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 0x558320de77f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00ede00 tx=0x7f6ba0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afbc5e6d30c7d3f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6baffff640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 0x7f6b980a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80107d0 tx=0x7f6ba8004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6b95ffb640 1 -- 172.21.15.49:0/2745934054 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00bc850 con 0x558320ddf150 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6b95ffb640 1 -- 172.21.15.49:0/2745934054 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba0034640 con 0x558320ddf150 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.246+0000 7f6b95ffb640 1 -- 172.21.15.49:0/2745934054 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0032ce0 con 0x558320ddf150 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2745934054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 msgr2=0x558320de77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 0x558320de77f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00ede00 tx=0x7f6ba0005f90 comp rx=0 tx=0).stop 2024-03-16T08:20:52.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2745934054 shutdown_connections 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320dea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320ddf150 0x558320de77f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2745934054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320ddec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2745934054 >> 172.21.15.49:0/2745934054 conn(0x558320dd8f60 msgr2=0x558320dd9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2745934054 shutdown_connections 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2745934054 wait complete. 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.247+0000 7f6bb8dce740 1 -- start start 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 Processor -- start 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320de5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320de7d30 0x558320de5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558320dea8e0 con 0x558320dde830 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558320cfe230 con 0x558320de7d30 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558320deaa60 con 0x558320ddf150 2024-03-16T08:20:52.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6bb8dce740 10 client.0 _ll_register_callbacks cb 0x558320d3a010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59406/0 (socket says 172.21.15.49:59406) 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2287825651 learned_addr learned my addr 172.21.15.49:0/2287825651 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320de5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baffff640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320de7d30 0x558320de5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2287825651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 msgr2=0x558320de5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320de5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320de7d30 msgr2=0x558320de5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320de7d30 0x558320de5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 -- 172.21.15.49:0/2287825651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558320d0d100 con 0x558320dde830 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c4f80 tx=0x7f6ba00c0fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2499b14aba2707e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.248+0000 7f6b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00bc670 con 0x558320dde830 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6ba00d0db0 con 0x558320dde830 2024-03-16T08:20:52.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558320d4f9d0 con 0x558320dde830 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6ba0032ce0 con 0x558320dde830 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 10 client.16539 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558320a563f0 con 0x558320dde830 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558320d4f9d0 con 0x558320dde830 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6ba003c030 con 0x558320dde830 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 10 client.16539 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 20 client.16539 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320df7c50 0x558320dfa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 4 client.16539 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558320d49d20 con 0x558320df7c50 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.249+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320df7c50 0x558320dfa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.250+0000 7f6b8effd640 10 client.16539 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.250+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320df7c50 0x558320dfa040 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1417d486046538 server_cookie=5b197913de539fd3 in_seq=0 out_seq=0 2024-03-16T08:20:52.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6b8effd640 1 -- 172.21.15.49:0/2287825651 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558320d49d20 con 0x558320df7c50 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6b8effd640 10 client.16539 handle_command_reply: tid=0 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 client.16539 shutdown 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 msgr2=0x558320de4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c4f80 tx=0x7f6ba00c0fe0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 shutdown_connections 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320df7c50 0x558320dfa040 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320de7d30 0x558320de5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320de5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6b8effd640 0 client.16539 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2287825651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320dde830 0x558320de4dc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 >> 172.21.15.49:0/2287825651 conn(0x558320dd8f60 msgr2=0x558320ddacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 shutdown_connections 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2287825651 wait complete. 2024-03-16T08:20:52.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.251+0000 7f6bb8dce740 20 client.16539 trim_cache size 0 max 16384 2024-03-16T08:20:52.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.264+0000 7f6bb8dce740 1 Processor -- start 2024-03-16T08:20:52.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.264+0000 7f6bb8dce740 1 -- start start 2024-03-16T08:20:52.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320de74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320df3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b1ef0 con 0x558320de7d30 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01af7c0 con 0x558320dde830 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ad610 con 0x558320ddf150 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59410/0 (socket says 172.21.15.49:59410) 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 -- 172.21.15.49:0/74296365 learned_addr learned my addr 172.21.15.49:0/74296365 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320de74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baeffd640 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320df3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 -- 172.21.15.49:0/74296365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 msgr2=0x558320df3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320df3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 -- 172.21.15.49:0/74296365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 msgr2=0x558320de74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320de74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 -- 172.21.15.49:0/74296365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558320dd4640 con 0x558320de7d30 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6baffff640 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800fac0 tx=0x7f6ba8013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda2806f38b5a290 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6b957fa640 1 -- 172.21.15.49:0/74296365 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8004d20 con 0x558320de7d30 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6b957fa640 1 -- 172.21.15.49:0/74296365 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba8005020 con 0x558320de7d30 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.265+0000 7f6b957fa640 1 -- 172.21.15.49:0/74296365 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba801c660 con 0x558320de7d30 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- 172.21.15.49:0/74296365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 msgr2=0x558320dd7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800fac0 tx=0x7f6ba8013030 comp rx=0 tx=0).stop 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- 172.21.15.49:0/74296365 shutdown_connections 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320de7d30 0x558320dd7e60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320ddf150 0x558320df3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/74296365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320de74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- 172.21.15.49:0/74296365 >> 172.21.15.49:0/74296365 conn(0x558320dd8f60 msgr2=0x558320dd7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- 172.21.15.49:0/74296365 shutdown_connections 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- 172.21.15.49:0/74296365 wait complete. 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.266+0000 7f6bb8dce740 1 -- start start 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 Processor -- start 2024-03-16T08:20:52.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320defe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320deb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01adc80 con 0x558320ddf150 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01af0f0 con 0x558320dde830 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ae350 con 0x558320de7d30 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6bb8dce740 10 client.0 _ll_register_callbacks cb 0x7f6ba8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59426/0 (socket says 172.21.15.49:59426) 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 -- 172.21.15.49:0/2272337209 learned_addr learned my addr 172.21.15.49:0/2272337209 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320defe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baffff640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320deb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 -- 172.21.15.49:0/2272337209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 msgr2=0x558320deb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320deb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 -- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 msgr2=0x558320defe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:52.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320defe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 -- 172.21.15.49:0/2272337209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558320b207d0 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6baeffd640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0075570 tx=0x7f6ba4010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cb5c4bc903e7ed server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.267+0000 7f6b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4020600 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6ba40207a0 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba8005020 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6ba402f050 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 10 client.16545 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558320e5de00 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba8005020 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6ba401f400 con 0x558320ddf150 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 10 client.16545 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:52.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 20 client.16545 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320e5e530 0x558320e60950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 4 client.16545 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558320cc76a0 con 0x558320e5e530 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.268+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320e5e530 0x558320e60950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6baf7fe640 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320e5e530 0x558320e60950 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5ba8121981722f server_cookie=e9823fdfa65996c3 in_seq=0 out_seq=0 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6b8effd640 10 client.16545 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6b8effd640 1 -- 172.21.15.49:0/2272337209 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558320cc76a0 con 0x558320e5e530 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6b8effd640 10 client.16545 handle_command_reply: tid=0 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 client.16545 shutdown 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 msgr2=0x558320deb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0075570 tx=0x7f6ba4010af0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 shutdown_connections 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558320e5e530 0x558320e60950 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558320de7d30 0x558320deb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558320ddf150 0x558320deb1f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 --2- 172.21.15.49:0/2272337209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558320dde830 0x558320defe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6b8effd640 0 client.16545 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 >> 172.21.15.49:0/2272337209 conn(0x558320dd8f60 msgr2=0x558320de5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 shutdown_connections 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 1 -- 172.21.15.49:0/2272337209 wait complete. 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.269+0000 7f6bb8dce740 20 client.16545 trim_cache size 0 max 16384 2024-03-16T08:20:52.277 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 msgr2=0x7f6b980a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 0x7f6b980a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80107d0 tx=0x7f6ba8004a40 comp rx=0 tx=0).stop 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 msgr2=0x7f6bb0073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6ba002c180 tx=0x7f6ba0004550 comp rx=0 tx=0).stop 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 shutdown_connections 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b980a2970 0x7f6b980a4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bb0107790 0x7f6bb0074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bb0102140 0x7f6bb0074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 --2- 172.21.15.49:0/782223153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb0101780 0x7f6bb0073ae0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 >> 172.21.15.49:0/782223153 conn(0x7f6bb00fd210 msgr2=0x7f6bb00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 shutdown_connections 2024-03-16T08:20:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:52.270+0000 7f6bb6113640 1 -- 172.21.15.49:0/782223153 wait complete. 2024-03-16T08:20:52.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:52 smithi129 ceph-mon[36561]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:52.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:52 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2723306737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:52.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:52 smithi154 ceph-mon[36471]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:52.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:52 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2723306737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:52.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:52 smithi049 ceph-mon[31942]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:52.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:52 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2723306737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:20:53.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:20:53.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:20:53.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 -- 172.21.15.49:0/1591547507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c10be20 msgr2=0x7fd94c10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 --2- 172.21.15.49:0/1591547507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c10be20 0x7fd94c10e290 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd944009ca0 tx=0x7fd94402c180 comp rx=0 tx=0).stop 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 -- 172.21.15.49:0/1591547507 shutdown_connections 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 --2- 172.21.15.49:0/1591547507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c10be20 0x7fd94c10e290 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 --2- 172.21.15.49:0/1591547507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c103370 0x7fd94c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 --2- 172.21.15.49:0/1591547507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c102a50 0x7fd94c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 -- 172.21.15.49:0/1591547507 >> 172.21.15.49:0/1591547507 conn(0x7fd94c077840 msgr2=0x7fd94c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 -- 172.21.15.49:0/1591547507 shutdown_connections 2024-03-16T08:20:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.410+0000 7fd951548640 1 -- 172.21.15.49:0/1591547507 wait complete. 2024-03-16T08:20:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 Processor -- start 2024-03-16T08:20:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 -- start start 2024-03-16T08:20:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c103370 0x7fd94c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c10be20 0x7fd94c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c111bf0 con 0x7fd94c102a50 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c111a70 con 0x7fd94c103370 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.411+0000 7fd951548640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c111d70 con 0x7fd94c10be20 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49012/0 (socket says 172.21.15.49:49012) 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 -- 172.21.15.49:0/80337994 learned_addr learned my addr 172.21.15.49:0/80337994 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c103370 0x7fd94c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c10be20 0x7fd94c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 -- 172.21.15.49:0/80337994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c10be20 msgr2=0x7fd94c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c10be20 0x7fd94c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 -- 172.21.15.49:0/80337994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c103370 msgr2=0x7fd94c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c103370 0x7fd94c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 -- 172.21.15.49:0/80337994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c19d0a0 con 0x7fd94c102a50 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd94affd640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd933fff640 1 -- 172.21.15.49:0/80337994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cd80 con 0x7fd94c102a50 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c19d390 con 0x7fd94c102a50 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd933fff640 1 -- 172.21.15.49:0/80337994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c010910 con 0x7fd94c102a50 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd933fff640 1 -- 172.21.15.49:0/80337994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0188f0 con 0x7fd94c102a50 2024-03-16T08:20:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.412+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1a4c20 con 0x7fd94c102a50 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.414+0000 7fd933fff640 1 -- 172.21.15.49:0/80337994 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd93c010ac0 con 0x7fd94c102a50 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 Processor -- start 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 -- start start 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbf9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc05190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55738bb5d0d0 con 0x55738bbfa710 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55738bb05c70 con 0x55738bbf9a30 2024-03-16T08:20:53.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd95420b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55738bb59170 con 0x55738bc02da0 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49020/0 (socket says 172.21.15.49:49020) 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94b7fe640 1 -- 172.21.15.49:0/3253091481 learned_addr learned my addr 172.21.15.49:0/3253091481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd933fff640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 0x7fd9340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc05190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94affd640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbf9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 0x7fd9340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.415+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3253091481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 msgr2=0x55738bbf9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbf9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3253091481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 msgr2=0x55738bbf9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd933fff640 1 -- 172.21.15.49:0/80337994 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd93c0cf1c0 con 0x7fd94c102a50 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3253091481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738b9397e0 con 0x55738bc02da0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc05190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd940009bd0 tx=0x7fd940011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30742ce2748d95f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 0x7fd9340a4e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd944009430 tx=0x7fd94402c180 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd9317fa640 1 -- 172.21.15.49:0/3253091481 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940004050 con 0x55738bc02da0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd9317fa640 1 -- 172.21.15.49:0/3253091481 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940005500 con 0x55738bc02da0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.416+0000 7fd9317fa640 1 -- 172.21.15.49:0/3253091481 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940004dd0 con 0x55738bc02da0 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- 172.21.15.49:0/3253091481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 msgr2=0x55738bc05190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc05190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd940009bd0 tx=0x7fd940011810 comp rx=0 tx=0).stop 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- 172.21.15.49:0/3253091481 shutdown_connections 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc05190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbf9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- 172.21.15.49:0/3253091481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bbf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- 172.21.15.49:0/3253091481 >> 172.21.15.49:0/3253091481 conn(0x55738bbc0c50 msgr2=0x55738bbf4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- 172.21.15.49:0/3253091481 shutdown_connections 2024-03-16T08:20:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- 172.21.15.49:0/3253091481 wait complete. 2024-03-16T08:20:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- start start 2024-03-16T08:20:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 Processor -- start 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc00750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55738bc05c10 con 0x55738bbfa710 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55738bb190d0 con 0x55738bbf9a30 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55738bc05d90 con 0x55738bc02da0 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.417+0000 7fd95420b740 10 client.0 _ll_register_callbacks cb 0x55738bae2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49022/0 (socket says 172.21.15.49:49022) 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 -- 172.21.15.49:0/1118823021 learned_addr learned my addr 172.21.15.49:0/1118823021 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94affd640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc00750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 -- 172.21.15.49:0/1118823021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 msgr2=0x55738bc00750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc00750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 -- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 msgr2=0x55738bbffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 -- 172.21.15.49:0/1118823021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738b999f70 con 0x55738bbfa710 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94affd640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd94001b040 tx=0x7fd940004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3ecb01e0091541 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940029070 con 0x55738bbfa710 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd940005140 con 0x55738bbfa710 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd940005500 con 0x55738bbfa710 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd940035050 con 0x55738bbfa710 2024-03-16T08:20:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd95420b740 10 client.16557 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.418+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55738b831170 con 0x55738bbfa710 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940005500 con 0x55738bbfa710 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd940003c90 con 0x55738bbfa710 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 10 client.16557 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 20 client.16557 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc12d40 0x55738bc15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 4 client.16557 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738bb6a610 con 0x55738bc12d40 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd94affd640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc12d40 0x55738bc15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd92eff5640 10 client.16557 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.419+0000 7fd94affd640 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc12d40 0x55738bc15130 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2743853ae6b46ce server_cookie=ef79c88d0ad7e789 in_seq=0 out_seq=0 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.420+0000 7fd92eff5640 1 -- 172.21.15.49:0/1118823021 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55738bb6a610 con 0x55738bc12d40 2024-03-16T08:20:53.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.420+0000 7fd92eff5640 10 client.16557 handle_command_reply: tid=0 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.420+0000 7fd95420b740 1 client.16557 shutdown 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 msgr2=0x55738bc00210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd94001b040 tx=0x7fd940004d40 comp rx=0 tx=0).stop 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 shutdown_connections 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc12d40 0x55738bc15130 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc00750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd92eff5640 0 client.16557 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc00210 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 --2- 172.21.15.49:0/1118823021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bbffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 >> 172.21.15.49:0/1118823021 conn(0x55738bbc0c50 msgr2=0x55738bbf61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 shutdown_connections 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 1 -- 172.21.15.49:0/1118823021 wait complete. 2024-03-16T08:20:53.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.421+0000 7fd95420b740 20 client.16557 trim_cache size 0 max 16384 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 Processor -- start 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 -- start start 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc1a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.438+0000 7fd95420b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1afc10 con 0x55738bbfa710 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd95420b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1ad4e0 con 0x55738bbf9a30 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd95420b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1ab310 con 0x55738bc02da0 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49024/0 (socket says 172.21.15.49:49024) 2024-03-16T08:20:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3711482016 learned_addr learned my addr 172.21.15.49:0/3711482016 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94affd640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc1a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3711482016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 msgr2=0x55738bc1a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc1a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3711482016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 msgr2=0x55738bc02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc02400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 -- 172.21.15.49:0/3711482016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738bc15760 con 0x55738bbfa710 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc1a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd940023480 tx=0x7fd940009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689d15fd1faa85af server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd930ff9640 1 -- 172.21.15.49:0/3711482016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940032070 con 0x55738bbfa710 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd930ff9640 1 -- 172.21.15.49:0/3711482016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940004630 con 0x55738bbfa710 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.439+0000 7fd930ff9640 1 -- 172.21.15.49:0/3711482016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940029050 con 0x55738bbfa710 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- 172.21.15.49:0/3711482016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 msgr2=0x55738bc0ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd940023480 tx=0x7fd940009680 comp rx=0 tx=0).stop 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- 172.21.15.49:0/3711482016 shutdown_connections 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc1a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc0ea50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- 172.21.15.49:0/3711482016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- 172.21.15.49:0/3711482016 >> 172.21.15.49:0/3711482016 conn(0x55738bbc0c50 msgr2=0x55738bbf6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- 172.21.15.49:0/3711482016 shutdown_connections 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- 172.21.15.49:0/3711482016 wait complete. 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 -- start start 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 Processor -- start 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.440+0000 7fd95420b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1ab9a0 con 0x55738bbfa710 2024-03-16T08:20:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1ace10 con 0x55738bbf9a30 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1ac070 con 0x55738bc02da0 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 10 client.0 _ll_register_callbacks cb 0x7fd93c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49026/0 (socket says 172.21.15.49:49026) 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 -- 172.21.15.49:0/20488766 learned_addr learned my addr 172.21.15.49:0/20488766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94affd640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94b7fe640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 -- 172.21.15.49:0/20488766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 msgr2=0x55738bc08590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc08590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 -- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 msgr2=0x55738bc0b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc0b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 -- 172.21.15.49:0/20488766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738bbf8a60 con 0x55738bbfa710 2024-03-16T08:20:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94affd640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd94a7fc640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd940004ae0 tx=0x7fd940004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b340afa7eaac00f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940015070 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd9400236a0 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd940004630 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd940032070 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 10 client.16563 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.441+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55738bc797f0 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940004630 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd9400035f0 con 0x55738bbfa710 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 10 client.16563 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 20 client.16563 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc79dd0 0x55738bc7c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 4 client.16563 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55738bb64960 con 0x55738bc79dd0 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd94affd640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc79dd0 0x55738bc7c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd92eff5640 10 client.16563 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.442+0000 7fd94affd640 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc79dd0 0x55738bc7c1f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9688a30b72bd58 server_cookie=f7d8ca5e943cab2e in_seq=0 out_seq=0 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd92eff5640 1 -- 172.21.15.49:0/20488766 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55738bb64960 con 0x55738bc79dd0 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd92eff5640 10 client.16563 handle_command_reply: tid=0 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 client.16563 shutdown 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 msgr2=0x55738bc08050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd940004ae0 tx=0x7fd940004db0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 shutdown_connections 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55738bc79dd0 0x55738bc7c1f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738bc02da0 0x55738bc08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55738bbfa710 0x55738bc08050 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd92eff5640 0 client.16563 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 --2- 172.21.15.49:0/20488766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55738bbf9a30 0x55738bc0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 >> 172.21.15.49:0/20488766 conn(0x55738bbc0c50 msgr2=0x55738bbffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 shutdown_connections 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 1 -- 172.21.15.49:0/20488766 wait complete. 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd95420b740 20 client.16563 trim_cache size 0 max 16384 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:20:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "64897305-0cf6-4c9f-9eea-212cecda13cf", 2024-03-16T08:20:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:20:53.453 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 msgr2=0x7fd9340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 0x7fd9340a4e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd944009430 tx=0x7fd94402c180 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 msgr2=0x7fd94c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 shutdown_connections 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd9340a2970 0x7fd9340a4e30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd94c10be20 0x7fd94c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd94c103370 0x7fd94c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.444+0000 7fd951548640 1 --2- 172.21.15.49:0/80337994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94c102a50 0x7fd94c1980b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.445+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 >> 172.21.15.49:0/80337994 conn(0x7fd94c077840 msgr2=0x7fd94c0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.445+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 shutdown_connections 2024-03-16T08:20:53.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.445+0000 7fd951548640 1 -- 172.21.15.49:0/80337994 wait complete. 2024-03-16T08:20:53.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:20:53.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 -- 172.21.15.49:0/3815626222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254102140 msgr2=0x7fa25410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3815626222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254102140 0x7fa25410e650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009ca0 tx=0x7fa24c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 -- 172.21.15.49:0/3815626222 shutdown_connections 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3815626222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254102140 0x7fa25410e650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3815626222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254101780 0x7fa254101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3815626222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254107790 0x7fa254107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 -- 172.21.15.49:0/3815626222 >> 172.21.15.49:0/3815626222 conn(0x7fa2540fd210 msgr2=0x7fa2540ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 -- 172.21.15.49:0/3815626222 shutdown_connections 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.570+0000 7fa25ab59640 1 -- 172.21.15.49:0/3815626222 wait complete. 2024-03-16T08:20:53.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 Processor -- start 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 -- start start 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254101780 0x7fa254073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254107790 0x7fa254074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa254111ba0 con 0x7fa254102140 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa254111a20 con 0x7fa254107790 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa25ab59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa254111d20 con 0x7fa254101780 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49040/0 (socket says 172.21.15.49:49040) 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa2588f5640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254101780 0x7fa254073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 -- 172.21.15.49:0/3303836669 learned_addr learned my addr 172.21.15.49:0/3303836669 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa2590f6640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254107790 0x7fa254074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 -- 172.21.15.49:0/3303836669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254101780 msgr2=0x7fa254073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254101780 0x7fa254073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 -- 172.21.15.49:0/3303836669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254107790 msgr2=0x7fa254074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254107790 0x7fa254074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.571+0000 7fa253fff640 1 -- 172.21.15.49:0/3303836669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2541a6520 con 0x7fa254102140 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa253fff640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa251ffb640 1 -- 172.21.15.49:0/3303836669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000cd80 con 0x7fa254102140 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2541a6810 con 0x7fa254102140 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa251ffb640 1 -- 172.21.15.49:0/3303836669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240010910 con 0x7fa254102140 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa251ffb640 1 -- 172.21.15.49:0/3303836669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400188f0 con 0x7fa254102140 2024-03-16T08:20:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.572+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541a6db0 con 0x7fa254102140 2024-03-16T08:20:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.574+0000 7fa251ffb640 1 -- 172.21.15.49:0/3303836669 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa240010ac0 con 0x7fa254102140 2024-03-16T08:20:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.574+0000 7fa25d814740 1 Processor -- start 2024-03-16T08:20:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.574+0000 7fa25d814740 1 -- start start 2024-03-16T08:20:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef292a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5629ef1eb500 con 0x5629ef287150 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629ef193be0 con 0x5629ef287a70 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa25d814740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629ef1e75a0 con 0x5629ef290650 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa251ffb640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 0x7fa2440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2590f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:56244/0 (socket says 172.21.15.49:56244) 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 -- 172.21.15.49:0/995609457 learned_addr learned my addr 172.21.15.49:0/995609457 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2590f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49048/0 (socket says 172.21.15.49:49048) 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2590f6640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 0x7fa2440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa251ffb640 1 -- 172.21.15.49:0/3303836669 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa2400cf1d0 con 0x7fa254102140 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa253fff640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef292a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 -- 172.21.15.49:0/995609457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 msgr2=0x5629ef292a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef292a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 -- 172.21.15.49:0/995609457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 msgr2=0x5629ef287530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 -- 172.21.15.49:0/995609457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ef1a74b0 con 0x5629ef287a70 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa253fff640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef292a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2590f6640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2590f6640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 0x7fa2440a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa24c004af0 tx=0x7fa24c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.575+0000 7fa2588f5640 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ca0 tx=0x7fa248011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb6cb3456d4c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa23effd640 1 -- 172.21.15.49:0/995609457 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24801e070 con 0x5629ef287a70 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa23effd640 1 -- 172.21.15.49:0/995609457 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248019b10 con 0x5629ef287a70 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa23effd640 1 -- 172.21.15.49:0/995609457 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800eac0 con 0x5629ef287a70 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 -- 172.21.15.49:0/995609457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 msgr2=0x5629ef290110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ca0 tx=0x7fa248011810 comp rx=0 tx=0).stop 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 -- 172.21.15.49:0/995609457 shutdown_connections 2024-03-16T08:20:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef292a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef290110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 --2- 172.21.15.49:0/995609457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287150 0x5629ef287530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 -- 172.21.15.49:0/995609457 >> 172.21.15.49:0/995609457 conn(0x5629ef281880 msgr2=0x5629ef281c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 -- 172.21.15.49:0/995609457 shutdown_connections 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.576+0000 7fa25d814740 1 -- 172.21.15.49:0/995609457 wait complete. 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 -- start start 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 Processor -- start 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef28dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 0x5629ef28e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5629ef293440 con 0x5629ef290650 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629ef1a7160 con 0x5629ef287a70 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629ef2935c0 con 0x5629ef287150 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa25d814740 10 client.0 _ll_register_callbacks cb 0x5629ef18b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa2590f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 0x5629ef28e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa253fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef28dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:44918/0 (socket says 172.21.15.49:44918) 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.577+0000 7fa2588f5640 1 -- 172.21.15.49:0/435568201 learned_addr learned my addr 172.21.15.49:0/435568201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 -- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 msgr2=0x5629ef28dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef28dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 -- 172.21.15.49:0/435568201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 msgr2=0x5629ef28e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 0x5629ef28e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 -- 172.21.15.49:0/435568201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629eefc97d0 con 0x5629ef287150 2024-03-16T08:20:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2590f6640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 0x5629ef28e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa2588f5640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa248009c70 tx=0x7fa248019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f7966e598eca28 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa233fff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248027020 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa248005250 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.578+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa248019b10 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa248032050 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 10 client.26458 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5629eeeff3f0 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248019b10 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2480046c0 con 0x5629ef287150 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 10 client.26458 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 20 client.26458 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef2a0620 0x5629ef2a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 4 client.26458 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:20:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629ef1f8a40 con 0x5629ef2a0620 2024-03-16T08:20:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.579+0000 7fa253fff640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef2a0620 0x5629ef2a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa233fff640 10 client.26458 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa253fff640 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef2a0620 0x5629ef2a2a10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356a1244b80457b server_cookie=2780fd2fecd96bb2 in_seq=0 out_seq=0 2024-03-16T08:20:53.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa233fff640 1 -- 172.21.15.49:0/435568201 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5629ef1f8a40 con 0x5629ef2a0620 2024-03-16T08:20:53.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa233fff640 10 client.26458 handle_command_reply: tid=0 2024-03-16T08:20:53.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 client.26458 shutdown 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 msgr2=0x5629ef28d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa248009c70 tx=0x7fa248019660 comp rx=0 tx=0).stop 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 shutdown_connections 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef2a0620 0x5629ef2a2a10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef290650 0x5629ef28e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287a70 0x5629ef28dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa233fff640 0 client.26458 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 --2- 172.21.15.49:0/435568201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef287150 0x5629ef28d6c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.580+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 >> 172.21.15.49:0/435568201 conn(0x5629ef281880 msgr2=0x5629ef2835d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.581+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 shutdown_connections 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.581+0000 7fa25d814740 1 -- 172.21.15.49:0/435568201 wait complete. 2024-03-16T08:20:53.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.581+0000 7fa25d814740 20 client.26458 trim_cache size 0 max 16384 2024-03-16T08:20:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 Processor -- start 2024-03-16T08:20:53.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 -- start start 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef28fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef2a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b1e30 con 0x5629ef287a70 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541af700 con 0x5629ef287150 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa25d814740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ad530 con 0x5629ef290650 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49060/0 (socket says 172.21.15.49:49060) 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa253fff640 1 -- 172.21.15.49:0/689093276 learned_addr learned my addr 172.21.15.49:0/689093276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa2588f5640 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef28fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.598+0000 7fa2590f6640 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef2a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 -- 172.21.15.49:0/689093276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 msgr2=0x5629ef2a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef2a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 -- 172.21.15.49:0/689093276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 msgr2=0x5629ef28fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef28fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 -- 172.21.15.49:0/689093276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629eeebf0f0 con 0x5629ef287a70 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa253fff640 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9c0 tx=0x7fa24006e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67d1a8124bd6ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa23e7fc640 1 -- 172.21.15.49:0/689093276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400cf030 con 0x5629ef287a70 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa23e7fc640 1 -- 172.21.15.49:0/689093276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2400a4020 con 0x5629ef287a70 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa23e7fc640 1 -- 172.21.15.49:0/689093276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400a3df0 con 0x5629ef287a70 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa25d814740 1 -- 172.21.15.49:0/689093276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 msgr2=0x5629ef29c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.599+0000 7fa25d814740 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9c0 tx=0x7fa24006e6d0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- 172.21.15.49:0/689093276 shutdown_connections 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef2a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef29c210 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- 172.21.15.49:0/689093276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef28fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- 172.21.15.49:0/689093276 >> 172.21.15.49:0/689093276 conn(0x5629ef281880 msgr2=0x5629ef28c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- 172.21.15.49:0/689093276 shutdown_connections 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- 172.21.15.49:0/689093276 wait complete. 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- start start 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 Processor -- start 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef293dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541adbc0 con 0x5629ef287a70 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541af030 con 0x5629ef287150 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ae290 con 0x5629ef290650 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.600+0000 7fa25d814740 10 client.0 _ll_register_callbacks cb 0x7fa24801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49066/0 (socket says 172.21.15.49:49066) 2024-03-16T08:20:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa2588f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:56274/0 (socket says 172.21.15.49:56274) 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 -- 172.21.15.49:0/3366384623 learned_addr learned my addr 172.21.15.49:0/3366384623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa2590f6640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef293dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 -- 172.21.15.49:0/3366384623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 msgr2=0x5629ef293dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef293dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 -- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 msgr2=0x5629ef2986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 -- 172.21.15.49:0/3366384623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ef1b29d0 con 0x5629ef287a70 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa2588f5640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa253fff640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2400097b0 tx=0x7fa240084ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f1f7b321b2d8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa233fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400ccdf0 con 0x5629ef287a70 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa2400a3b90 con 0x5629ef287a70 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2400a4020 con 0x5629ef287a70 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2400a3d90 con 0x5629ef287a70 2024-03-16T08:20:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa25d814740 10 client.16581 fetch_fsmap learned FSMap version 24 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5629ef3062c0 con 0x5629ef287a70 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.601+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2400a4020 con 0x5629ef287a70 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa240097680 con 0x5629ef287a70 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 10 client.16581 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 20 client.16581 populate_metadata read hostname 'smithi049' 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef306980 0x5629ef308d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 4 client.16581 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629ef1f2d90 con 0x5629ef306980 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa2588f5640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef306980 0x5629ef308d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa2588f5640 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef306980 0x5629ef308d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708a5516d1891955 server_cookie=a253f82b64467120 in_seq=0 out_seq=0 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.602+0000 7fa233fff640 10 client.16581 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa233fff640 1 -- 172.21.15.49:0/3366384623 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5629ef1f2d90 con 0x5629ef306980 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa233fff640 10 client.16581 handle_command_reply: tid=0 2024-03-16T08:20:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 client.16581 shutdown 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 msgr2=0x5629ef298c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2400097b0 tx=0x7fa240084ef0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 shutdown_connections 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5629ef306980 0x5629ef308d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629ef290650 0x5629ef293dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5629ef287a70 0x5629ef298c30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa233fff640 0 client.16581 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 --2- 172.21.15.49:0/3366384623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629ef287150 0x5629ef2986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 >> 172.21.15.49:0/3366384623 conn(0x5629ef281880 msgr2=0x5629ef280070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 shutdown_connections 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 1 -- 172.21.15.49:0/3366384623 wait complete. 2024-03-16T08:20:53.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.603+0000 7fa25d814740 20 client.16581 trim_cache size 0 max 16384 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "64897305-0cf6-4c9f-9eea-212cecda13cf": { 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "64897305-0cf6-4c9f-9eea-212cecda13cf", 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:20:53.614 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 msgr2=0x7fa2440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 0x7fa2440a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa24c004af0 tx=0x7fa24c005c50 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 msgr2=0x7fa254074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 shutdown_connections 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa2440a2970 0x7fa2440a4e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa254107790 0x7fa254074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa254102140 0x7fa254074020 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 --2- 172.21.15.49:0/3303836669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa254101780 0x7fa254073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 >> 172.21.15.49:0/3303836669 conn(0x7fa2540fd210 msgr2=0x7fa2540fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 shutdown_connections 2024-03-16T08:20:53.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:20:53.604+0000 7fa25ab59640 1 -- 172.21.15.49:0/3303836669 wait complete. 2024-03-16T08:20:53.616 INFO:tasks.cephfs.filesystem:scrub status for tag:64897305-0cf6-4c9f-9eea-212cecda13cf - {'path': '/', 'tag': '64897305-0cf6-4c9f-9eea-212cecda13cf', 'options': 'recursive,force'} 2024-03-16T08:20:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:54 smithi129 ceph-mon[36561]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:54 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:20:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:54 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:54 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:20:54.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:54 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:54 smithi154 ceph-mon[36471]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:54 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:20:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:54 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:54 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:20:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:54 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:54.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:54 smithi049 ceph-mon[31942]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:20:54.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:54 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:20:54.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:54 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:54.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:54 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:20:54.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:54 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:20:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:56 smithi129 ceph-mon[36561]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:56.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:56 smithi154 ceph-mon[36471]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:56.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:56 smithi049 ceph-mon[31942]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:20:57.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:57 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:20:57.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:57 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:20:57.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:57 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:20:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:20:58 smithi129 ceph-mon[36561]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:20:58 smithi154 ceph-mon[36471]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:20:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:20:58 smithi049 ceph-mon[31942]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:00 smithi129 ceph-mon[36561]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:00 smithi154 ceph-mon[36471]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:00 smithi049 ceph-mon[31942]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:02 smithi129 ceph-mon[36561]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:02 smithi154 ceph-mon[36471]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:02 smithi049 ceph-mon[31942]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:04 smithi129 ceph-mon[36561]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:04 smithi154 ceph-mon[36471]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:04 smithi049 ceph-mon[31942]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:06 smithi129 ceph-mon[36561]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:21:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:06 smithi154 ceph-mon[36471]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:21:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:06 smithi049 ceph-mon[31942]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:21:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:08 smithi129 ceph-mon[36561]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:08.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:08 smithi154 ceph-mon[36471]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:08 smithi049 ceph-mon[31942]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:10 smithi154 ceph-mon[36471]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:10.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:10 smithi049 ceph-mon[31942]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:10.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:10 smithi129 ceph-mon[36561]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:12 smithi154 ceph-mon[36471]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:12.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:12 smithi049 ceph-mon[31942]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:12.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:12 smithi129 ceph-mon[36561]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:14.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:14 smithi154 ceph-mon[36471]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:14 smithi049 ceph-mon[31942]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:14 smithi129 ceph-mon[36561]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:21:16.411 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:16 smithi129 ceph-mon[36561]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:16 smithi154 ceph-mon[36471]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:16 smithi049 ceph-mon[31942]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:21:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:18 smithi154 ceph-mon[36471]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:21:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:18 smithi049 ceph-mon[31942]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:21:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:18 smithi129 ceph-mon[36561]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:21:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:20 smithi154 ceph-mon[36471]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:21:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:20 smithi049 ceph-mon[31942]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:21:20.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:20 smithi129 ceph-mon[36561]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:21:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:22 smithi154 ceph-mon[36471]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:22 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:21:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:22 smithi049 ceph-mon[31942]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:22 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:21:22.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:22 smithi129 ceph-mon[36561]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:22.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:22 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:21:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:21:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:21:23.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:23 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:21:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:21:23.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:21:23.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:23 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:21:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:21:23.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:21:23.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:23 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:21:23.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:21:23.617 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:21:23.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- 172.21.15.49:0/784364903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8101750 msgr2=0x7ff9d8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 --2- 172.21.15.49:0/784364903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8101750 0x7ff9d8101bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009890 tx=0x7ff9cc02bb70 comp rx=0 tx=0).stop 2024-03-16T08:21:23.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- 172.21.15.49:0/784364903 shutdown_connections 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 --2- 172.21.15.49:0/784364903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8102110 0x7ff9d810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 --2- 172.21.15.49:0/784364903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8101750 0x7ff9d8101bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 --2- 172.21.15.49:0/784364903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 0x7ff9d8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- 172.21.15.49:0/784364903 >> 172.21.15.49:0/784364903 conn(0x7ff9d80fd240 msgr2=0x7ff9d80ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- 172.21.15.49:0/784364903 shutdown_connections 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- 172.21.15.49:0/784364903 wait complete. 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 Processor -- start 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.741+0000 7ff9deeff640 1 -- start start 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8102110 0x7ff9d81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 0x7ff9d819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8111b30 con 0x7ff9d8101750 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81119b0 con 0x7ff9d8107760 2024-03-16T08:21:23.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8111cb0 con 0x7ff9d8102110 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51834/0 (socket says 172.21.15.49:51834) 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/1441641152 learned_addr learned my addr 172.21.15.49:0/1441641152 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8102110 0x7ff9d81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 0x7ff9d819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/1441641152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8102110 msgr2=0x7ff9d81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8102110 0x7ff9d81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/1441641152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 msgr2=0x7ff9d819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 0x7ff9d819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/1441641152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d819d080 con 0x7ff9d8101750 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c420 tx=0x7ff9d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9d5ffb640 1 -- 172.21.15.49:0/1441641152 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0003970 con 0x7ff9d8101750 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d819d370 con 0x7ff9d8101750 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9d5ffb640 1 -- 172.21.15.49:0/1441641152 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d0004020 con 0x7ff9d8101750 2024-03-16T08:21:23.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.742+0000 7ff9d5ffb640 1 -- 172.21.15.49:0/1441641152 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000bad0 con 0x7ff9d8101750 2024-03-16T08:21:23.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.743+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a4c00 con 0x7ff9d8101750 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9d5ffb640 1 -- 172.21.15.49:0/1441641152 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ff9d001d020 con 0x7ff9d8101750 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 Processor -- start 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 -- start start 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9d5ffb640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 0x7ff9c00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 0x55a67f5c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a67f518500 con 0x55a67f5b3e80 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a67f4c0be0 con 0x55a67f5b47a0 2024-03-16T08:21:23.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a67f5145a0 con 0x55a67f5c1a60 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9d5ffb640 1 -- 172.21.15.49:0/1441641152 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff9d00cec50 con 0x7ff9d8101750 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.745+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 0x7ff9c00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51838/0 (socket says 172.21.15.49:51838) 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3122325970 learned_addr learned my addr 172.21.15.49:0/3122325970 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 0x55a67f5c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3122325970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 msgr2=0x55a67f5c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3122325970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 msgr2=0x55a67f5c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 0x55a67f5c1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3122325970 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67f4d44b0 con 0x55a67f5b3e80 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 0x55a67f5c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 0x7ff9c00a4e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009710 tx=0x7ff9c80098b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0097c90 tx=0x7ff9d009af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8760a4559609979f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9beffd640 1 -- 172.21.15.49:0/3122325970 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d009b9c0 con 0x55a67f5b3e80 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9beffd640 1 -- 172.21.15.49:0/3122325970 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d000bc70 con 0x55a67f5b3e80 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.746+0000 7ff9beffd640 1 -- 172.21.15.49:0/3122325970 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0049f80 con 0x55a67f5b3e80 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3122325970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 msgr2=0x55a67f5b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0097c90 tx=0x7ff9d009af30 comp rx=0 tx=0).stop 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3122325970 shutdown_connections 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b47a0 0x55a67f5c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3122325970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5b4260 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3122325970 >> 172.21.15.49:0/3122325970 conn(0x55a67f5ae880 msgr2=0x55a67f5aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3122325970 shutdown_connections 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3122325970 wait complete. 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 -- start start 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.747+0000 7ff9e1bc9740 1 Processor -- start 2024-03-16T08:21:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a67f5c4820 con 0x55a67f5b3e80 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a67f4d4160 con 0x55a67f5c1a60 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a67f5c49a0 con 0x55a67f5b47a0 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9e1bc9740 10 client.0 _ll_register_callbacks cb 0x55a67f4b8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51852/0 (socket says 172.21.15.49:51852) 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/532814553 learned_addr learned my addr 172.21.15.49:0/532814553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/532814553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 msgr2=0x55a67f5bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 msgr2=0x55a67f5bf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5bf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/532814553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67f2f67d0 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff9d004c6d0 tx=0x7ff9d009b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2e5d2a6917839f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.748+0000 7ff9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d009f410 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff9d00bb0b0 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d000bc70 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff9d009b9c0 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 10 client.16593 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a67f22c3f0 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d000bc70 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff9d000b040 con 0x55a67f5b3e80 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 10 client.16593 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 20 client.16593 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:23.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f5d1970 0x55a67f5d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 4 client.16593 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67f525a40 con 0x55a67f5d1970 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.749+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f5d1970 0x55a67f5d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.750+0000 7ff9b7fff640 10 client.16593 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.750+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f5d1970 0x55a67f5d3d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9691c2fc1d235c9 server_cookie=482ddd39d577afc1 in_seq=0 out_seq=0 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9b7fff640 1 -- 172.21.15.49:0/532814553 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a67f525a40 con 0x55a67f5d1970 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9b7fff640 10 client.16593 handle_command_reply: tid=0 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 client.16593 shutdown 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 msgr2=0x55a67f5beb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff9d004c6d0 tx=0x7ff9d009b1c0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 shutdown_connections 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f5d1970 0x55a67f5d3d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/532814553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5beb20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 >> 172.21.15.49:0/532814553 conn(0x55a67f5ae880 msgr2=0x55a67f5b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9b7fff640 0 client.16593 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 shutdown_connections 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/532814553 wait complete. 2024-03-16T08:21:23.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.751+0000 7ff9e1bc9740 20 client.16593 trim_cache size 0 max 16384 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.769+0000 7ff9e1bc9740 1 Processor -- start 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.769+0000 7ff9e1bc9740 1 -- start start 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 0x55a67f5ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81adae0 con 0x55a67f5b47a0 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ab3b0 con 0x55a67f5b3e80 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81a9200 con 0x55a67f5c1a60 2024-03-16T08:21:23.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51868/0 (socket says 172.21.15.49:51868) 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 -- 172.21.15.49:0/931394554 learned_addr learned my addr 172.21.15.49:0/931394554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 0x55a67f5ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 -- 172.21.15.49:0/931394554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 msgr2=0x55a67f5cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 -- 172.21.15.49:0/931394554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 msgr2=0x55a67f5ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 0x55a67f5ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 -- 172.21.15.49:0/931394554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67f5c41c0 con 0x55a67f5b47a0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 0x55a67f5ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.770+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005e00 tx=0x7ff9cc02c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847ca97096bfdf76 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9be7fc640 1 -- 172.21.15.49:0/931394554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc04e070 con 0x55a67f5b47a0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9be7fc640 1 -- 172.21.15.49:0/931394554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc034440 con 0x55a67f5b47a0 2024-03-16T08:21:23.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9be7fc640 1 -- 172.21.15.49:0/931394554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc039040 con 0x55a67f5b47a0 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/931394554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 msgr2=0x55a67f5b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005e00 tx=0x7ff9cc02c310 comp rx=0 tx=0).stop 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/931394554 shutdown_connections 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5c1a60 0x55a67f5cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b47a0 0x55a67f5b9e80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/931394554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5b3e80 0x55a67f5ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/931394554 >> 172.21.15.49:0/931394554 conn(0x55a67f5ae880 msgr2=0x55a67f5bf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/931394554 shutdown_connections 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.771+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/931394554 wait complete. 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 -- start start 2024-03-16T08:21:23.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 Processor -- start 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81a9870 con 0x55a67f5b3e80 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81aace0 con 0x55a67f5c1a60 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81a9f40 con 0x55a67f5b47a0 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9e1bc9740 10 client.0 _ll_register_callbacks cb 0x7ff9c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9dcc9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51884/0 (socket says 172.21.15.49:51884) 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3863437568 learned_addr learned my addr 172.21.15.49:0/3863437568 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9dd49c640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.772+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3863437568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 msgr2=0x55a67f5c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 msgr2=0x55a67f5c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 -- 172.21.15.49:0/3863437568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67f1ec0f0 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9dcc9b640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff9d009b250 tx=0x7ff9d00997d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1445f5b1aa929f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d009f410 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff9d00bb0b0 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc034440 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff9d0049c80 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9e1bc9740 10 client.16599 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a67f4df9d0 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc034440 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.773+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff9d00a35d0 con 0x55a67f5b3e80 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 10 client.16599 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:23.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 20 client.16599 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f637c60 0x55a67f63a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 4 client.16599 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a67f51fd90 con 0x55a67f637c60 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f637c60 0x55a67f63a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9d7fff640 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f637c60 0x55a67f63a050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6b87f5bbabb52f server_cookie=2b96fa655bc701da in_seq=0 out_seq=0 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9b7fff640 10 client.16599 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9b7fff640 1 -- 172.21.15.49:0/3863437568 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a67f51fd90 con 0x55a67f637c60 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9b7fff640 10 client.16599 handle_command_reply: tid=0 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 client.16599 shutdown 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 msgr2=0x55a67f5c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff9d009b250 tx=0x7ff9d00997d0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 shutdown_connections 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55a67f637c60 0x55a67f63a050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a67f5c1a60 0x55a67f5c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a67f5b47a0 0x55a67f5c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9b7fff640 0 client.16599 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 --2- 172.21.15.49:0/3863437568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a67f5b3e80 0x55a67f5c5470 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.774+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 >> 172.21.15.49:0/3863437568 conn(0x55a67f5ae880 msgr2=0x55a67f5bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 shutdown_connections 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9e1bc9740 1 -- 172.21.15.49:0/3863437568 wait complete. 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9e1bc9740 20 client.16599 trim_cache size 0 max 16384 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:21:23.782 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:21:23.783 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:23.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 msgr2=0x7ff9c00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 0x7ff9c00a4e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009710 tx=0x7ff9c80098b0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 msgr2=0x7ff9d8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c420 tx=0x7ff9d0009f90 comp rx=0 tx=0).stop 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 shutdown_connections 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff9c00a2970 0x7ff9c00a4e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9d8107760 0x7ff9d819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d8102110 0x7ff9d81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 --2- 172.21.15.49:0/1441641152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d8101750 0x7ff9d8198090 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 >> 172.21.15.49:0/1441641152 conn(0x7ff9d80fd240 msgr2=0x7ff9d80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 shutdown_connections 2024-03-16T08:21:23.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.775+0000 7ff9deeff640 1 -- 172.21.15.49:0/1441641152 wait complete. 2024-03-16T08:21:23.788 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:21:23.788 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:21:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- 172.21.15.49:0/3033795493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec06a830 msgr2=0x7f2fec10e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/3033795493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec06a830 0x7f2fec10e630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009ca0 tx=0x7f2fe402c180 comp rx=0 tx=0).stop 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- 172.21.15.49:0/3033795493 shutdown_connections 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/3033795493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec06a830 0x7f2fec10e630 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/3033795493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec069e90 0x7f2fec06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/3033795493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec1077e0 0x7f2fec107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- 172.21.15.49:0/3033795493 >> 172.21.15.49:0/3033795493 conn(0x7f2fec077780 msgr2=0x7f2fec077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- 172.21.15.49:0/3033795493 shutdown_connections 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- 172.21.15.49:0/3033795493 wait complete. 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 Processor -- start 2024-03-16T08:21:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.895+0000 7f2ff13f4640 1 -- start start 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec06a830 0x7f2fec198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec1077e0 0x7f2fec19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec111ba0 con 0x7f2fec069e90 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec111a20 con 0x7f2fec1077e0 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec111d20 con 0x7f2fec06a830 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51900/0 (socket says 172.21.15.49:51900) 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 -- 172.21.15.49:0/173512936 learned_addr learned my addr 172.21.15.49:0/173512936 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2fea7fc640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec06a830 0x7f2fec198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feb7fe640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec1077e0 0x7f2fec19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 -- 172.21.15.49:0/173512936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec06a830 msgr2=0x7f2fec198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec06a830 0x7f2fec198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 -- 172.21.15.49:0/173512936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec1077e0 msgr2=0x7f2fec19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec1077e0 0x7f2fec19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 -- 172.21.15.49:0/173512936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec19cfe0 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2feaffd640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00ea60 tx=0x7f2fdc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00cde0 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec19d2d0 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fdc0035f0 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc010670 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.896+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec1a4b60 con 0x7f2fec069e90 2024-03-16T08:21:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.898+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec102f00 con 0x7f2fec069e90 2024-03-16T08:21:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.899+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2fdc0037a0 con 0x7f2fec069e90 2024-03-16T08:21:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.899+0000 7f2fd3fff640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 0x7f2fd40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.899+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2fdc0cf090 con 0x7f2fec069e90 2024-03-16T08:21:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.899+0000 7f2fea7fc640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 0x7f2fd40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.900+0000 7f2fea7fc640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 0x7f2fd40a4e30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fec077d60 tx=0x7f2fe0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:23.903+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2fdc09bf70 con 0x7f2fec069e90 2024-03-16T08:21:24.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.127+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fec068a20 con 0x7f2fec069e90 2024-03-16T08:21:24.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.128+0000 7f2fd3fff640 1 -- 172.21.15.49:0/173512936 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f2fdc09b870 con 0x7f2fec069e90 2024-03-16T08:21:24.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:21:24.129 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:21:24.129 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.129+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 msgr2=0x7f2fd40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.129+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 0x7f2fd40a4e30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fec077d60 tx=0x7f2fe0005f90 comp rx=0 tx=0).stop 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.129+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 msgr2=0x7f2fec198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.129+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00ea60 tx=0x7f2fdc00ef20 comp rx=0 tx=0).stop 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 shutdown_connections 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2fd40a2970 0x7f2fd40a4e30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fec1077e0 0x7f2fec19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fec06a830 0x7f2fec198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 --2- 172.21.15.49:0/173512936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fec069e90 0x7f2fec198040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 >> 172.21.15.49:0/173512936 conn(0x7f2fec077780 msgr2=0x7f2fec0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 shutdown_connections 2024-03-16T08:21:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.130+0000 7f2ff13f4640 1 -- 172.21.15.49:0/173512936 wait complete. 2024-03-16T08:21:24.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:21:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 -- 172.21.15.49:0/3977098102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8100700 msgr2=0x7fa5c8100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3977098102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8100700 0x7fa5c8100b80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009930 tx=0x7fa5c402bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 -- 172.21.15.49:0/3977098102 shutdown_connections 2024-03-16T08:21:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3977098102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c81010c0 0x7fa5c810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3977098102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8100700 0x7fa5c8100b80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3977098102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 0x7fa5c8106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 -- 172.21.15.49:0/3977098102 >> 172.21.15.49:0/3977098102 conn(0x7fa5c8077840 msgr2=0x7fa5c8077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 -- 172.21.15.49:0/3977098102 shutdown_connections 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.252+0000 7fa5d0577640 1 -- 172.21.15.49:0/3977098102 wait complete. 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 Processor -- start 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 -- start start 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c8100700 0x7fa5c81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 0x7fa5c819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8110b20 con 0x7fa5c81010c0 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81109a0 con 0x7fa5c8106710 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5d0577640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8110ca0 con 0x7fa5c8100700 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51912/0 (socket says 172.21.15.49:51912) 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 -- 172.21.15.49:0/3737789434 learned_addr learned my addr 172.21.15.49:0/3737789434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5ce313640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c8100700 0x7fa5c81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 0x7fa5c819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 -- 172.21.15.49:0/3737789434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c8100700 msgr2=0x7fa5c81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c8100700 0x7fa5c81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 -- 172.21.15.49:0/3737789434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 msgr2=0x7fa5c819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 0x7fa5c819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.253+0000 7fa5cdb12640 1 -- 172.21.15.49:0/3737789434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819d150 con 0x7fa5c81010c0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009230 tx=0x7fa5c402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5bb7fe640 1 -- 172.21.15.49:0/3737789434 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4041070 con 0x7fa5c81010c0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819d3e0 con 0x7fa5c81010c0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5bb7fe640 1 -- 172.21.15.49:0/3737789434 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c4004450 con 0x7fa5c81010c0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5bb7fe640 1 -- 172.21.15.49:0/3737789434 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003de0 con 0x7fa5c81010c0 2024-03-16T08:21:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.254+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a4cd0 con 0x7fa5c81010c0 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 Processor -- start 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5bb7fe640 1 -- 172.21.15.49:0/3737789434 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa5c4044070 con 0x7fa5c81010c0 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 -- start start 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 0x563a72d3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 0x563a72d447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563a72c9f490 con 0x563a72d3c150 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563a72c47c90 con 0x563a72d44d30 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563a72c9b530 con 0x563a72d3b830 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5ce313640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 0x563a72d447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 0x563a72d3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 0x563a72d3bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:57536/0 (socket says 172.21.15.49:57536) 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5cdb12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.257+0000 7fa5ceb14640 1 -- 172.21.15.49:0/3566672560 learned_addr learned my addr 172.21.15.49:0/3566672560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5bb7fe640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 0x7fa5b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 -- 172.21.15.49:0/3566672560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 msgr2=0x563a72d3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 0x563a72d3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 -- 172.21.15.49:0/3566672560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 msgr2=0x563a72d47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 -- 172.21.15.49:0/3566672560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a72d02500 con 0x563a72d3c150 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 0x7fa5b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5bb7fe640 1 -- 172.21.15.49:0/3737789434 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa5c40f0670 con 0x7fa5c81010c0 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ce313640 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 0x563a72d447f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009c20 tx=0x7fa5bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd49e008253503d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5b8bfe640 1 -- 172.21.15.49:0/3566672560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc004050 con 0x563a72d3c150 2024-03-16T08:21:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 0x7fa5b40a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0003bf0 tx=0x7fa5c0004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5b8bfe640 1 -- 172.21.15.49:0/3566672560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc005500 con 0x563a72d3c150 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.258+0000 7fa5b8bfe640 1 -- 172.21.15.49:0/3566672560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc004d60 con 0x563a72d3c150 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/3566672560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 msgr2=0x563a72d447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 0x563a72d447f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009c20 tx=0x7fa5bc011810 comp rx=0 tx=0).stop 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/3566672560 shutdown_connections 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3c150 0x563a72d447f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/3566672560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3b830 0x563a72d3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/3566672560 >> 172.21.15.49:0/3566672560 conn(0x563a72d35f60 msgr2=0x563a72d36340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/3566672560 shutdown_connections 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/3566672560 wait complete. 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.259+0000 7fa5d2dfd740 1 -- start start 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 Processor -- start 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d41d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563a72d47b20 con 0x563a72d44d30 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563a72c5b230 con 0x563a72d3b830 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563a72d47ca0 con 0x563a72d3c150 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5d2dfd740 10 client.0 _ll_register_callbacks cb 0x563a72c97010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ce313640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d41d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51932/0 (socket says 172.21.15.49:51932) 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5cdb12640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 -- 172.21.15.49:0/2806612445 learned_addr learned my addr 172.21.15.49:0/2806612445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 -- 172.21.15.49:0/2806612445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 msgr2=0x563a72d422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 -- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 msgr2=0x563a72d41d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d41d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.260+0000 7fa5ceb14640 1 -- 172.21.15.49:0/2806612445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a72c6a100 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0010b90 tx=0x7fa5c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ec10d74edbdd5a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c002e060 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa5c001dd80 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5bc005500 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5c0032410 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 10 client.16617 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563a729b33f0 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc005500 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa5c001b670 con 0x563a72d44d30 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 10 client.16617 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 10 client.16617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 20 client.16617 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72d55130 0x563a72d57560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 4 client.16617 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.261+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a72cac9d0 con 0x563a72d55130 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.262+0000 7fa5ce313640 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72d55130 0x563a72d57560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.262+0000 7fa5ce313640 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72d55130 0x563a72d57560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fa69dea48fb042e server_cookie=99aae8ece09b04ad in_seq=0 out_seq=0 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.262+0000 7fa5b1ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.262+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/2806612445 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563a72cac9d0 con 0x563a72d55130 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.262+0000 7fa5b1ffb640 10 client.16617 handle_command_reply: tid=0 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 client.16617 shutdown 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 msgr2=0x563a72d427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0010b90 tx=0x7fa5c0004ab0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 shutdown_connections 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72d55130 0x563a72d57560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d427f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/2806612445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d41d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 >> 172.21.15.49:0/2806612445 conn(0x563a72d35f60 msgr2=0x563a72d37cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5b1ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 shutdown_connections 2024-03-16T08:21:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/2806612445 wait complete. 2024-03-16T08:21:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.263+0000 7fa5d2dfd740 20 client.16617 trim_cache size 0 max 16384 2024-03-16T08:21:24.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 Processor -- start 2024-03-16T08:21:24.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 -- start start 2024-03-16T08:21:24.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81afd30 con 0x563a72d44d30 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81ad600 con 0x563a72d3b830 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.276+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81ab430 con 0x563a72d3c150 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51940/0 (socket says 172.21.15.49:51940) 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5cdb12640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ce313640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 -- 172.21.15.49:0/959356417 learned_addr learned my addr 172.21.15.49:0/959356417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 -- 172.21.15.49:0/959356417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 msgr2=0x563a72d508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 -- 172.21.15.49:0/959356417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 msgr2=0x563a72d444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 -- 172.21.15.49:0/959356417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a72a7d7d0 con 0x563a72d44d30 2024-03-16T08:21:24.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0004570 tx=0x7fa5c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ba56fe1064759a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5b3fff640 1 -- 172.21.15.49:0/959356417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003970 con 0x563a72d44d30 2024-03-16T08:21:24.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5b3fff640 1 -- 172.21.15.49:0/959356417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c001beb0 con 0x563a72d44d30 2024-03-16T08:21:24.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.277+0000 7fa5b3fff640 1 -- 172.21.15.49:0/959356417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0032b20 con 0x563a72d44d30 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/959356417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 msgr2=0x563a72d37aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0004570 tx=0x7fa5c0009580 comp rx=0 tx=0).stop 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/959356417 shutdown_connections 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d44d30 0x563a72d37aa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/959356417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d3b830 0x563a72d444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/959356417 >> 172.21.15.49:0/959356417 conn(0x563a72d35f60 msgr2=0x563a72d37390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/959356417 shutdown_connections 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/959356417 wait complete. 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- start start 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 Processor -- start 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81abac0 con 0x563a72d3b830 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81acf30 con 0x563a72d44d30 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81ac190 con 0x563a72d3c150 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.278+0000 7fa5d2dfd740 10 client.0 _ll_register_callbacks cb 0x7fa5c0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51950/0 (socket says 172.21.15.49:51950) 2024-03-16T08:21:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 -- 172.21.15.49:0/23846593 learned_addr learned my addr 172.21.15.49:0/23846593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ceb14640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 -- 172.21.15.49:0/23846593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 msgr2=0x563a72d48eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d48eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 -- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 msgr2=0x563a72d493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 -- 172.21.15.49:0/23846593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a72dc2f10 con 0x563a72d3b830 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5ce313640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc011d10 tx=0x7fa5bc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73b714c822c4277 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc030030 con 0x563a72d3b830 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa5bc02a030 con 0x563a72d3b830 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a72ca6d20 con 0x563a72d3b830 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5bc03a050 con 0x563a72d3b830 2024-03-16T08:21:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5d2dfd740 10 client.16623 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.279+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563a72dc4210 con 0x563a72d3b830 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563a72ca6d20 con 0x563a72d3b830 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa5bc004320 con 0x563a72d3b830 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 10 client.16623 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 10 client.16623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 20 client.16623 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72dc4820 0x563a72dc6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 4 client.16623 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a72c246a0 con 0x563a72dc4820 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72dc4820 0x563a72dc6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5cdb12640 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72dc4820 0x563a72dc6c40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=945d351a15200b3e server_cookie=ff3531157d8208a1 in_seq=0 out_seq=0 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.280+0000 7fa5b1ffb640 10 client.16623 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5b1ffb640 1 -- 172.21.15.49:0/23846593 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a72c246a0 con 0x563a72dc4820 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5b1ffb640 10 client.16623 handle_command_reply: tid=0 2024-03-16T08:21:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 client.16623 shutdown 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 msgr2=0x563a72d48970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc011d10 tx=0x7fa5bc009680 comp rx=0 tx=0).stop 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 shutdown_connections 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x563a72dc4820 0x563a72dc6c40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a72d44d30 0x563a72d493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563a72d3c150 0x563a72d48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5b1ffb640 0 client.16623 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 --2- 172.21.15.49:0/23846593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a72d3b830 0x563a72d48970 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 >> 172.21.15.49:0/23846593 conn(0x563a72d35f60 msgr2=0x563a72d34fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 shutdown_connections 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 1 -- 172.21.15.49:0/23846593 wait complete. 2024-03-16T08:21:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d2dfd740 20 client.16623 trim_cache size 0 max 16384 2024-03-16T08:21:24.289 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 msgr2=0x7fa5b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 0x7fa5b40a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0003bf0 tx=0x7fa5c0004a40 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 msgr2=0x7fa5c81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009230 tx=0x7fa5c402cb30 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 shutdown_connections 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa5b40a2970 0x7fa5b40a4e30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c8106710 0x7fa5c819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c81010c0 0x7fa5c81986f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 --2- 172.21.15.49:0/3737789434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5c8100700 0x7fa5c81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 >> 172.21.15.49:0/3737789434 conn(0x7fa5c8077840 msgr2=0x7fa5c806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 shutdown_connections 2024-03-16T08:21:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.281+0000 7fa5d0577640 1 -- 172.21.15.49:0/3737789434 wait complete. 2024-03-16T08:21:24.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 -- 172.21.15.49:0/1704894974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4100700 msgr2=0x7fefc4100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1704894974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4100700 0x7fefc4100b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009930 tx=0x7fefb402bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 -- 172.21.15.49:0/1704894974 shutdown_connections 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1704894974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc41010c0 0x7fefc410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1704894974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4100700 0x7fefc4100b80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1704894974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc4106710 0x7fefc4106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 -- 172.21.15.49:0/1704894974 >> 172.21.15.49:0/1704894974 conn(0x7fefc4077840 msgr2=0x7fefc4077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 -- 172.21.15.49:0/1704894974 shutdown_connections 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.403+0000 7fefcbe97640 1 -- 172.21.15.49:0/1704894974 wait complete. 2024-03-16T08:21:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 Processor -- start 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 -- start start 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 0x7fefc4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 0x7fefc4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4110b20 con 0x7fefc4106710 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41109a0 con 0x7fefc4100700 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefcbe97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4110ca0 con 0x7fefc41010c0 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefca434640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefca434640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51972/0 (socket says 172.21.15.49:51972) 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefca434640 1 -- 172.21.15.49:0/1726325533 learned_addr learned my addr 172.21.15.49:0/1726325533 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefc9432640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 0x7fefc4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.404+0000 7fefc9c33640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 0x7fefc4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 -- 172.21.15.49:0/1726325533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 msgr2=0x7fefc4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 0x7fefc4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 -- 172.21.15.49:0/1726325533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 msgr2=0x7fefc4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 0x7fefc4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 -- 172.21.15.49:0/1726325533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41a4500 con 0x7fefc4106710 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefc9432640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 0x7fefc4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefc9c33640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 0x7fefc4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefca434640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00eb10 tx=0x7fefbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefbaffd640 1 -- 172.21.15.49:0/1726325533 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004b20 con 0x7fefc4106710 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefbaffd640 1 -- 172.21.15.49:0/1726325533 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc004cc0 con 0x7fefc4106710 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a46d0 con 0x7fefc4106710 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefbaffd640 1 -- 172.21.15.49:0/1726325533 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc010760 con 0x7fefc4106710 2024-03-16T08:21:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.405+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a4ba0 con 0x7fefc4106710 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.407+0000 7fefbaffd640 1 -- 172.21.15.49:0/1726325533 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fefbc005020 con 0x7fefc4106710 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 Processor -- start 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 -- start start 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 0x56342c290c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c29c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56342c1f4490 con 0x56342c299d30 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56342c19cc90 con 0x56342c291150 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefcea82740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56342c1f0530 con 0x56342c290830 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefbaffd640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 0x7fefb00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c29c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefc9c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefc9432640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 0x56342c290c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefc9c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:32910/0 (socket says 172.21.15.49:32910) 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.408+0000 7fefc9c33640 1 -- 172.21.15.49:0/482590791 learned_addr learned my addr 172.21.15.49:0/482590791 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefca434640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 0x7fefb00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 -- 172.21.15.49:0/482590791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 msgr2=0x56342c2997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 -- 172.21.15.49:0/482590791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 msgr2=0x56342c29c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c29c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 -- 172.21.15.49:0/482590791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342c257500 con 0x56342c290830 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefbaffd640 1 -- 172.21.15.49:0/1726325533 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fefbc0cf3d0 con 0x7fefc4106710 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9c33640 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefc9432640 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 0x56342c290c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009230 tx=0x7fefb4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcfcc1a8782c0340 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefaffff640 1 -- 172.21.15.49:0/482590791 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb402ca80 con 0x56342c290830 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefca434640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 0x7fefb00a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefbc097ea0 tx=0x7fefbc00efd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefaffff640 1 -- 172.21.15.49:0/482590791 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb4004ca0 con 0x56342c290830 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.409+0000 7fefaffff640 1 -- 172.21.15.49:0/482590791 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4005570 con 0x56342c290830 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- 172.21.15.49:0/482590791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 msgr2=0x56342c290c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 0x56342c290c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009230 tx=0x7fefb4005fd0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- 172.21.15.49:0/482590791 shutdown_connections 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c29c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c291150 0x56342c2997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 --2- 172.21.15.49:0/482590791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c290830 0x56342c290c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- 172.21.15.49:0/482590791 >> 172.21.15.49:0/482590791 conn(0x56342c28af60 msgr2=0x56342c28b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- 172.21.15.49:0/482590791 shutdown_connections 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- 172.21.15.49:0/482590791 wait complete. 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 -- start start 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.410+0000 7fefcea82740 1 Processor -- start 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c297300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c297840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56342c29cac0 con 0x56342c290830 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56342c1b0230 con 0x56342c299d30 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56342c29cc40 con 0x56342c291150 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefcea82740 10 client.0 _ll_register_callbacks cb 0x56342c1ec010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51984/0 (socket says 172.21.15.49:51984) 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 -- 172.21.15.49:0/1311468910 learned_addr learned my addr 172.21.15.49:0/1311468910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9c33640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c297300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c297840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 -- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 msgr2=0x56342c297300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c297300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 -- 172.21.15.49:0/1311468910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 msgr2=0x56342c297840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c297840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 -- 172.21.15.49:0/1311468910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342c1bf100 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefc9432640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c00 tx=0x7fefb40051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f860414721d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4044070 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.411+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fefb4004040 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb4004ca0 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fefb4004240 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 10 client.16635 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56342bf083f0 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb4004ca0 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fefb404e470 con 0x56342c290830 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 10 client.16635 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 10 client.16635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 20 client.16635 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c2a9f40 0x56342c2ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 4 client.16635 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342c2019d0 con 0x56342c2a9f40 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.412+0000 7fefc9c33640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c2a9f40 0x56342c2ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefad7fa640 10 client.16635 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefc9c33640 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c2a9f40 0x56342c2ac330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e71bbce97fb3d731 server_cookie=b3a960cf9b82eff5 in_seq=0 out_seq=0 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefad7fa640 1 -- 172.21.15.49:0/1311468910 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56342c2019d0 con 0x56342c2a9f40 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefad7fa640 10 client.16635 handle_command_reply: tid=0 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefcea82740 1 client.16635 shutdown 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 msgr2=0x56342c296dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.413+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c00 tx=0x7fefb40051b0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 shutdown_connections 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c2a9f40 0x56342c2ac330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c297840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c297300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefad7fa640 0 client.16635 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 --2- 172.21.15.49:0/1311468910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c296dc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 >> 172.21.15.49:0/1311468910 conn(0x56342c28af60 msgr2=0x56342c28ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 shutdown_connections 2024-03-16T08:21:24.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 1 -- 172.21.15.49:0/1311468910 wait complete. 2024-03-16T08:21:24.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.414+0000 7fefcea82740 20 client.16635 trim_cache size 0 max 16384 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 Processor -- start 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 -- start start 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c289e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c2af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41afc30 con 0x56342c299d30 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ad500 con 0x56342c290830 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefcea82740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ab330 con 0x56342c291150 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c2af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:32920/0 (socket says 172.21.15.49:32920) 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 -- 172.21.15.49:0/3197326419 learned_addr learned my addr 172.21.15.49:0/3197326419 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9c33640 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c289e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 -- 172.21.15.49:0/3197326419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 msgr2=0x56342c289e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c289e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 -- 172.21.15.49:0/3197326419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 msgr2=0x56342c2af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c2af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 -- 172.21.15.49:0/3197326419 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342bfd27d0 con 0x56342c290830 2024-03-16T08:21:24.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.428+0000 7fefc9432640 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefb404e890 tx=0x7fefb4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c348c5dab2bd52 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefaf7fe640 1 -- 172.21.15.49:0/3197326419 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4044040 con 0x56342c290830 2024-03-16T08:21:24.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefaf7fe640 1 -- 172.21.15.49:0/3197326419 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb405e070 con 0x56342c290830 2024-03-16T08:21:24.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefaf7fe640 1 -- 172.21.15.49:0/3197326419 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb404eb00 con 0x56342c290830 2024-03-16T08:21:24.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 -- 172.21.15.49:0/3197326419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 msgr2=0x56342c2994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefb404e890 tx=0x7fefb4005fd0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 -- 172.21.15.49:0/3197326419 shutdown_connections 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c299d30 0x56342c2af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c289e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 --2- 172.21.15.49:0/3197326419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c290830 0x56342c2994f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 -- 172.21.15.49:0/3197326419 >> 172.21.15.49:0/3197326419 conn(0x56342c28af60 msgr2=0x56342c289750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 -- 172.21.15.49:0/3197326419 shutdown_connections 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.429+0000 7fefcea82740 1 -- 172.21.15.49:0/3197326419 wait complete. 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 -- start start 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 Processor -- start 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c2ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c2ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ab9c0 con 0x56342c290830 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ace30 con 0x56342c299d30 2024-03-16T08:21:24.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ac090 con 0x56342c291150 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefcea82740 10 client.0 _ll_register_callbacks cb 0x7fefb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefc9c33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c2ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefc9432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51998/0 (socket says 172.21.15.49:51998) 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefc9432640 1 -- 172.21.15.49:0/341386371 learned_addr learned my addr 172.21.15.49:0/341386371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.430+0000 7fefc9432640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c2ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 -- 172.21.15.49:0/341386371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 msgr2=0x56342c2ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c2ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 -- 172.21.15.49:0/341386371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 msgr2=0x56342c2acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 -- 172.21.15.49:0/341386371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342c30f130 con 0x56342c291150 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9432640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefc9c33640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c2ad470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56342c29cd70 tx=0x7fefc0013a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1cd0a7611a2cfcd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:21:24.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0026060 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fefc0005220 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb405e070 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fefc00044d0 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefcea82740 10 client.26524 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56342c310490 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb405e070 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.431+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fefc0022180 con 0x56342c291150 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 10 client.26524 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 10 client.26524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 20 client.26524 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c310be0 0x56342c313000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 4 client.26524 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56342c1fbd20 con 0x56342c310be0 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefc9432640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c310be0 0x56342c313000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefc9432640 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c310be0 0x56342c313000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef6f9509cf88faa1 server_cookie=269284aaa03e43a6 in_seq=0 out_seq=0 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefad7fa640 10 client.26524 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefad7fa640 1 -- 172.21.15.49:0/341386371 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56342c1fbd20 con 0x56342c310be0 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefad7fa640 10 client.26524 handle_command_reply: tid=0 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 client.26524 shutdown 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 msgr2=0x56342c2ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c2ad470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56342c29cd70 tx=0x7fefc0013a50 comp rx=0 tx=0).stop 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 shutdown_connections 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56342c310be0 0x56342c313000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c299d30 0x56342c2ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c291150 0x56342c2ad470 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefad7fa640 0 client.26524 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.432+0000 7fefcea82740 1 --2- 172.21.15.49:0/341386371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c290830 0x56342c2acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 >> 172.21.15.49:0/341386371 conn(0x56342c28af60 msgr2=0x56342c289fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 shutdown_connections 2024-03-16T08:21:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcea82740 1 -- 172.21.15.49:0/341386371 wait complete. 2024-03-16T08:21:24.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcea82740 20 client.26524 trim_cache size 0 max 16384 2024-03-16T08:21:24.441 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 msgr2=0x7fefb00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 0x7fefb00a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefbc097ea0 tx=0x7fefbc00efd0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 msgr2=0x7fefc41a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00eb10 tx=0x7fefbc00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 shutdown_connections 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fefb00a2970 0x7fefb00a4e30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefc4106710 0x7fefc41a3fc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefc41010c0 0x7fefc4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 --2- 172.21.15.49:0/1726325533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefc4100700 0x7fefc4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 >> 172.21.15.49:0/1726325533 conn(0x7fefc4077840 msgr2=0x7fefc4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 shutdown_connections 2024-03-16T08:21:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.433+0000 7fefcbe97640 1 -- 172.21.15.49:0/1726325533 wait complete. 2024-03-16T08:21:24.445 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 -- 172.21.15.49:0/907111620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0107790 msgr2=0x7f2ee0107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/907111620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0107790 0x7f2ee0107b70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009930 tx=0x7f2edc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 -- 172.21.15.49:0/907111620 shutdown_connections 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/907111620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0102140 0x7f2ee010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/907111620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0101780 0x7f2ee0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/907111620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0107790 0x7f2ee0107b70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 -- 172.21.15.49:0/907111620 >> 172.21.15.49:0/907111620 conn(0x7f2ee00fd210 msgr2=0x7f2ee00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 -- 172.21.15.49:0/907111620 shutdown_connections 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.554+0000 7f2ee86bf640 1 -- 172.21.15.49:0/907111620 wait complete. 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 Processor -- start 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 -- start start 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0101780 0x7f2ee0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0107790 0x7f2ee019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0111b60 con 0x7f2ee0102140 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01119e0 con 0x7f2ee0107790 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee86bf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0111ce0 con 0x7f2ee0101780 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52014/0 (socket says 172.21.15.49:52014) 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/828387325 learned_addr learned my addr 172.21.15.49:0/828387325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.555+0000 7f2ee645b640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0101780 0x7f2ee0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0107790 0x7f2ee019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0101780 msgr2=0x7f2ee0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0101780 0x7f2ee0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0107790 msgr2=0x7f2ee019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0107790 0x7f2ee019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee019d070 con 0x7f2ee0102140 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000ea60 tx=0x7f2ed000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ed77fe640 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000cde0 con 0x7f2ee0102140 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ed77fe640 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed00035f0 con 0x7f2ee0102140 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee019d360 con 0x7f2ee0102140 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ed77fe640 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0010670 con 0x7f2ee0102140 2024-03-16T08:21:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.556+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a4bf0 con 0x7f2ee0102140 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.558+0000 7f2ed77fe640 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2ed00037a0 con 0x7f2ee0102140 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.558+0000 7f2eeb382740 1 Processor -- start 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 -- start start 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cfb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a84c56490 con 0x559a84cfbd30 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a84bfec90 con 0x559a84cf2830 2024-03-16T08:21:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2eeb382740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559a84c52530 con 0x559a84cf3150 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee6c5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52030/0 (socket says 172.21.15.49:52030) 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/3473466674 learned_addr learned my addr 172.21.15.49:0/3473466674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee645b640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cfb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ed77fe640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 0x7f2ecc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/3473466674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 msgr2=0x559a84cfb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cfb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/3473466674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 msgr2=0x559a84cf2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.559+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ed77fe640 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2ed00cf060 con 0x7f2ee0102140 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ee5c5a640 1 -- 172.21.15.49:0/3473466674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a84cb9500 con 0x559a84cfbd30 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ee645b640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cfb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 0x7f2ecc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cc740 tx=0x7f2ed0017f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffdd5db63d0ef7b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ed4ff9640 1 -- 172.21.15.49:0/3473466674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed009b250 con 0x559a84cfbd30 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ed4ff9640 1 -- 172.21.15.49:0/3473466674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed009be10 con 0x559a84cfbd30 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.560+0000 7f2ed4ff9640 1 -- 172.21.15.49:0/3473466674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0086110 con 0x559a84cfbd30 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- 172.21.15.49:0/3473466674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 msgr2=0x559a84cfe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 0x7f2ecc0a4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009fd0 tx=0x7f2ed8004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cc740 tx=0x7f2ed0017f70 comp rx=0 tx=0).stop 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- 172.21.15.49:0/3473466674 shutdown_connections 2024-03-16T08:21:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84cfe120 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cfb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- 172.21.15.49:0/3473466674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- 172.21.15.49:0/3473466674 >> 172.21.15.49:0/3473466674 conn(0x559a84cecf60 msgr2=0x559a84ced340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- 172.21.15.49:0/3473466674 shutdown_connections 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- 172.21.15.49:0/3473466674 wait complete. 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- start start 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 Processor -- start 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cfbd30 0x559a84cf9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.561+0000 7f2eeb382740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a84cfeac0 con 0x559a84cf3150 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2eeb382740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a84c12230 con 0x559a84cf2830 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2eeb382740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559a84cfec40 con 0x559a84cfbd30 2024-03-16T08:21:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2eeb382740 10 client.0 _ll_register_callbacks cb 0x559a84c4e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:32960/0 (socket says 172.21.15.49:32960) 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52042/0 (socket says 172.21.15.49:52042) 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 -- 172.21.15.49:0/2784508627 learned_addr learned my addr 172.21.15.49:0/2784508627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cfbd30 0x559a84cf9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 -- 172.21.15.49:0/2784508627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cfbd30 msgr2=0x559a84cf9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cfbd30 0x559a84cf9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 -- 172.21.15.49:0/2784508627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 msgr2=0x559a84cf92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 -- 172.21.15.49:0/2784508627 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a84c21100 con 0x559a84cf2830 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2edc036040 tx=0x7f2edc009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0ee453ef27bc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004460 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2edc0046c0 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.562+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed009be10 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2edc0048c0 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 10 client.26534 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a8496a3f0 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed009be10 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2edc0037b0 con 0x559a84cf2830 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 10 client.26534 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 20 client.26534 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d0bfd0 0x559a84d0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 4 client.26534 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a84c639d0 con 0x559a84d0bfd0 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d0bfd0 0x559a84d0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d0bfd0 0x559a84d0e440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d8920cf95a9ba2 server_cookie=4e8210db403a476e in_seq=0 out_seq=0 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.563+0000 7f2ec9ffb640 10 client.26534 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2784508627 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559a84c639d0 con 0x559a84d0bfd0 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2ec9ffb640 10 client.26534 handle_command_reply: tid=0 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 client.26534 shutdown 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 msgr2=0x559a84cf8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2edc036040 tx=0x7f2edc009600 comp rx=0 tx=0).stop 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 shutdown_connections 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d0bfd0 0x559a84d0e440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cfbd30 0x559a84cf9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf3150 0x559a84cf92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2784508627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cf8d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 >> 172.21.15.49:0/2784508627 conn(0x559a84cecf60 msgr2=0x559a84ceecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2ec9ffb640 0 client.26534 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 shutdown_connections 2024-03-16T08:21:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 1 -- 172.21.15.49:0/2784508627 wait complete. 2024-03-16T08:21:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.564+0000 7f2eeb382740 20 client.26534 trim_cache size 0 max 16384 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.577+0000 7f2eeb382740 1 Processor -- start 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 -- start start 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cfb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84d07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01afdd0 con 0x559a84cfbd30 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ad6a0 con 0x559a84cf2830 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2eeb382740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ab4d0 con 0x559a84cf3150 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52044/0 (socket says 172.21.15.49:52044) 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 -- 172.21.15.49:0/1986647357 learned_addr learned my addr 172.21.15.49:0/1986647357 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee645b640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cfb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84d07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 -- 172.21.15.49:0/1986647357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 msgr2=0x559a84d07860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84d07860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 -- 172.21.15.49:0/1986647357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 msgr2=0x559a84cfb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cfb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 -- 172.21.15.49:0/1986647357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a84a347d0 con 0x559a84cfbd30 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee645b640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cfb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800f9a0 tx=0x7f2ed8013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58ef0bf00860b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.578+0000 7f2ecbfff640 1 -- 172.21.15.49:0/1986647357 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801d0d0 con 0x559a84cfbd30 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2ecbfff640 1 -- 172.21.15.49:0/1986647357 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed801d270 con 0x559a84cfbd30 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2ecbfff640 1 -- 172.21.15.49:0/1986647357 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8005280 con 0x559a84cfbd30 2024-03-16T08:21:24.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- 172.21.15.49:0/1986647357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 msgr2=0x559a84d13d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800f9a0 tx=0x7f2ed8013030 comp rx=0 tx=0).stop 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- 172.21.15.49:0/1986647357 shutdown_connections 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cfbd30 0x559a84d13d10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84d07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 --2- 172.21.15.49:0/1986647357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cf2830 0x559a84cfb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- 172.21.15.49:0/1986647357 >> 172.21.15.49:0/1986647357 conn(0x559a84cecf60 msgr2=0x559a84cf8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- 172.21.15.49:0/1986647357 shutdown_connections 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- 172.21.15.49:0/1986647357 wait complete. 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 -- start start 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.579+0000 7f2eeb382740 1 Processor -- start 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cfbd30 0x559a84cff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01abb60 con 0x559a84cf2830 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01acfd0 con 0x559a84cfbd30 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ac230 con 0x559a84cf3150 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 10 client.0 _ll_register_callbacks cb 0x7f2edc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52048/0 (socket says 172.21.15.49:52048) 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 -- 172.21.15.49:0/2162688345 learned_addr learned my addr 172.21.15.49:0/2162688345 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee5c5a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee6c5c640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cfbd30 0x559a84cff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 -- 172.21.15.49:0/2162688345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 msgr2=0x559a84cff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 -- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cfbd30 msgr2=0x559a84cff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cfbd30 0x559a84cff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 -- 172.21.15.49:0/2162688345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a84d00190 con 0x559a84cf2830 2024-03-16T08:21:24.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ee645b640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009850 tx=0x7f2edc004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabc41b2b00d1803 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc005850 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2edc004470 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.580+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a84c5dd20 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2edc060050 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 10 client.16659 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a84d722b0 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559a84c5dd20 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2edc005450 con 0x559a84cf2830 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 10 client.16659 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 20 client.16659 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d72a30 0x559a84d74e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 4 client.16659 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a84bdb6a0 con 0x559a84d72a30 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.581+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d72a30 0x559a84d74e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ec9ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ee5c5a640 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d72a30 0x559a84d74e50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1836a1aef4bf49d server_cookie=58e1fbf0554046ca in_seq=0 out_seq=0 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ec9ffb640 1 -- 172.21.15.49:0/2162688345 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a84bdb6a0 con 0x559a84d72a30 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ec9ffb640 10 client.16659 handle_command_reply: tid=0 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 client.16659 shutdown 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 msgr2=0x559a84d04090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009850 tx=0x7f2edc004c70 comp rx=0 tx=0).stop 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 shutdown_connections 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559a84d72a30 0x559a84d74e50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a84cfbd30 0x559a84cff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559a84cf3150 0x559a84cff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 --2- 172.21.15.49:0/2162688345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a84cf2830 0x559a84d04090 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ec9ffb640 0 client.16659 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:24.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 >> 172.21.15.49:0/2162688345 conn(0x559a84cecf60 msgr2=0x559a84cf95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 shutdown_connections 2024-03-16T08:21:24.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 1 -- 172.21.15.49:0/2162688345 wait complete. 2024-03-16T08:21:24.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2eeb382740 20 client.16659 trim_cache size 0 max 16384 2024-03-16T08:21:24.591 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 msgr2=0x7f2ecc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.582+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 0x7f2ecc0a4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009fd0 tx=0x7f2ed8004a40 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 msgr2=0x7f2ee01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000ea60 tx=0x7f2ed000ef20 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 shutdown_connections 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2ecc0a29f0 0x7f2ecc0a4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee0107790 0x7f2ee019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee0102140 0x7f2ee01985c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ee0101780 0x7f2ee0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 >> 172.21.15.49:0/828387325 conn(0x7f2ee00fd210 msgr2=0x7f2ee00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 shutdown_connections 2024-03-16T08:21:24.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:24.583+0000 7f2ee86bf640 1 -- 172.21.15.49:0/828387325 wait complete. 2024-03-16T08:21:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:24 smithi154 ceph-mon[36471]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:24.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:24 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/173512936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:24.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:24 smithi049 ceph-mon[31942]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:24.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:24 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/173512936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:24.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:24 smithi129 ceph-mon[36561]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:24.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:24 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/173512936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:25.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:21:25.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:21:25.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:21:25.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 -- 172.21.15.49:0/2861840701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468101780 msgr2=0x7fe468101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 --2- 172.21.15.49:0/2861840701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468101780 0x7fe468101c00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009930 tx=0x7fe45c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:25.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 -- 172.21.15.49:0/2861840701 shutdown_connections 2024-03-16T08:21:25.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 --2- 172.21.15.49:0/2861840701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468102140 0x7fe46810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 --2- 172.21.15.49:0/2861840701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468101780 0x7fe468101c00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 --2- 172.21.15.49:0/2861840701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468107790 0x7fe468107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 -- 172.21.15.49:0/2861840701 >> 172.21.15.49:0/2861840701 conn(0x7fe4680fd210 msgr2=0x7fe4680ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 -- 172.21.15.49:0/2861840701 shutdown_connections 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.714+0000 7fe46e47e640 1 -- 172.21.15.49:0/2861840701 wait complete. 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 Processor -- start 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 -- start start 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 0x7fe468073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 0x7fe468074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe468111ba0 con 0x7fe468102140 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe468111a20 con 0x7fe468107790 2024-03-16T08:21:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.715+0000 7fe46e47e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe468111d20 con 0x7fe468101780 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52056/0 (socket says 172.21.15.49:52056) 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 -- 172.21.15.49:0/3518235789 learned_addr learned my addr 172.21.15.49:0/3518235789 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe467fff640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 0x7fe468073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 0x7fe468074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 -- 172.21.15.49:0/3518235789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 msgr2=0x7fe468073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 0x7fe468073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 -- 172.21.15.49:0/3518235789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 msgr2=0x7fe468074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 0x7fe468074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 -- 172.21.15.49:0/3518235789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4681a6600 con 0x7fe468102140 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 0x7fe468074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe467fff640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 0x7fe468073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4677fe640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009230 tx=0x7fe45c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4657fa640 1 -- 172.21.15.49:0/3518235789 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c041070 con 0x7fe468102140 2024-03-16T08:21:25.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681a6890 con 0x7fe468102140 2024-03-16T08:21:25.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4657fa640 1 -- 172.21.15.49:0/3518235789 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45c004450 con 0x7fe468102140 2024-03-16T08:21:25.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe4657fa640 1 -- 172.21.15.49:0/3518235789 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c003e50 con 0x7fe468102140 2024-03-16T08:21:25.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.716+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681a6d80 con 0x7fe468102140 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe4657fa640 1 -- 172.21.15.49:0/3518235789 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fe45c044070 con 0x7fe468102140 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 Processor -- start 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 -- start start 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9c9a30 0x55985a9c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55985a92d0d0 con 0x55985a9ca710 2024-03-16T08:21:25.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55985a8d5c70 con 0x55985a9d2da0 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe471063740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55985a929170 con 0x55985a9c9a30 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe4657fa640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 0x7fe4500a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe46ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe46ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52064/0 (socket says 172.21.15.49:52064) 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe46ca1b640 1 -- 172.21.15.49:0/1071988886 learned_addr learned my addr 172.21.15.49:0/1071988886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe4677fe640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe467fff640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9c9a30 0x55985a9c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 0x7fe4500a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.719+0000 7fe4657fa640 1 -- 172.21.15.49:0/3518235789 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe45c0f05f0 con 0x7fe468102140 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 -- 172.21.15.49:0/1071988886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9c9a30 msgr2=0x55985a9c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9c9a30 0x55985a9c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 -- 172.21.15.49:0/1071988886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 msgr2=0x55985a9d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9d5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 -- 172.21.15.49:0/1071988886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985a7097e0 con 0x55985a9ca710 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe4677fe640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe46000f5c0 tx=0x7fe46000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e5fa6f01a3eb86 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 0x7fe4500a4fd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe4600099e0 tx=0x7fe46001e000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe44e7fc640 1 -- 172.21.15.49:0/1071988886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46002b070 con 0x55985a9ca710 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe44e7fc640 1 -- 172.21.15.49:0/1071988886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe460003e50 con 0x55985a9ca710 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.720+0000 7fe44e7fc640 1 -- 172.21.15.49:0/1071988886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460022070 con 0x55985a9ca710 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- 172.21.15.49:0/1071988886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 msgr2=0x55985a9c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe46000f5c0 tx=0x7fe46000fa80 comp rx=0 tx=0).stop 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- 172.21.15.49:0/1071988886 shutdown_connections 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9c9a30 0x55985a9c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 --2- 172.21.15.49:0/1071988886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9c9460 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- 172.21.15.49:0/1071988886 >> 172.21.15.49:0/1071988886 conn(0x55985a990c50 msgr2=0x55985a9c4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- 172.21.15.49:0/1071988886 shutdown_connections 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- 172.21.15.49:0/1071988886 wait complete. 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 -- start start 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.721+0000 7fe471063740 1 Processor -- start 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9d2da0 0x55985a9d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55985a9d5bb0 con 0x55985a9ca710 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55985a8e90d0 con 0x55985a9c9a30 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55985a9d5d30 con 0x55985a9d2da0 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 10 client.0 _ll_register_callbacks cb 0x55985a8b2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52070/0 (socket says 172.21.15.49:52070) 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 -- 172.21.15.49:0/2876039484 learned_addr learned my addr 172.21.15.49:0/2876039484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe467fff640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9d2da0 0x55985a9d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 -- 172.21.15.49:0/2876039484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9d2da0 msgr2=0x55985a9d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9d2da0 0x55985a9d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 -- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 msgr2=0x55985a9cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 -- 172.21.15.49:0/2876039484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985a769f70 con 0x55985a9ca710 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4677fe640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0edf00 tx=0x7fe45c003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73de5728451a9ee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4477fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c04b040 con 0x55985a9ca710 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe45c0ee010 con 0x55985a9ca710 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.722+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe460003e50 con 0x55985a9ca710 2024-03-16T08:21:25.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe45c0bc7c0 con 0x55985a9ca710 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 10 client.16671 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55985a601170 con 0x55985a9ca710 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe460003e50 con 0x55985a9ca710 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe45c04b1e0 con 0x55985a9ca710 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 10 client.16671 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 20 client.16671 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985a9e2df0 0x55985a9e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 4 client.16671 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55985a93a610 con 0x55985a9e2df0 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.723+0000 7fe467fff640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985a9e2df0 0x55985a9e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.724+0000 7fe4477fe640 10 client.16671 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.724+0000 7fe467fff640 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985a9e2df0 0x55985a9e5240 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826658a710f56693 server_cookie=fa580699e8d68971 in_seq=0 out_seq=0 2024-03-16T08:21:25.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.724+0000 7fe4477fe640 1 -- 172.21.15.49:0/2876039484 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55985a93a610 con 0x55985a9e2df0 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.724+0000 7fe4477fe640 10 client.16671 handle_command_reply: tid=0 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.724+0000 7fe471063740 1 client.16671 shutdown 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 msgr2=0x55985a9d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0edf00 tx=0x7fe45c003920 comp rx=0 tx=0).stop 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 shutdown_connections 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985a9e2df0 0x55985a9e5240 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9d2da0 0x55985a9d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9ca710 0x55985a9d02a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 --2- 172.21.15.49:0/2876039484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 >> 172.21.15.49:0/2876039484 conn(0x55985a990c50 msgr2=0x55985a9c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe4477fe640 0 client.16671 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 shutdown_connections 2024-03-16T08:21:25.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 1 -- 172.21.15.49:0/2876039484 wait complete. 2024-03-16T08:21:25.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.725+0000 7fe471063740 20 client.16671 trim_cache size 0 max 16384 2024-03-16T08:21:25.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.742+0000 7fe471063740 1 Processor -- start 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.742+0000 7fe471063740 1 -- start start 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b1d80 con 0x55985a9c9a30 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681af650 con 0x55985a9d2da0 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe471063740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681ad480 con 0x55985a9ca710 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52074/0 (socket says 172.21.15.49:52074) 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 -- 172.21.15.49:0/2687207034 learned_addr learned my addr 172.21.15.49:0/2687207034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe4677fe640 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 -- 172.21.15.49:0/2687207034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 msgr2=0x55985a9de9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9de9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 -- 172.21.15.49:0/2687207034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 msgr2=0x55985a9c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 -- 172.21.15.49:0/2687207034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985a9c8a60 con 0x55985a9c9a30 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe467fff640 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe458009fd0 tx=0x7fe458005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de02d5409ff55c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe44dffb640 1 -- 172.21.15.49:0/2687207034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45801f630 con 0x55985a9c9a30 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe44dffb640 1 -- 172.21.15.49:0/2687207034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45801f7d0 con 0x55985a9c9a30 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.743+0000 7fe44dffb640 1 -- 172.21.15.49:0/2687207034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458010690 con 0x55985a9c9a30 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- 172.21.15.49:0/2687207034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 msgr2=0x55985a9d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe458009fd0 tx=0x7fe458005f90 comp rx=0 tx=0).stop 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- 172.21.15.49:0/2687207034 shutdown_connections 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9d2da0 0x55985a9c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9de9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 --2- 172.21.15.49:0/2687207034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9c9a30 0x55985a9d2490 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- 172.21.15.49:0/2687207034 >> 172.21.15.49:0/2687207034 conn(0x55985a990c50 msgr2=0x55985a9c5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- 172.21.15.49:0/2687207034 shutdown_connections 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- 172.21.15.49:0/2687207034 wait complete. 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.744+0000 7fe471063740 1 -- start start 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 Processor -- start 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681adb10 con 0x55985a9d2da0 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681aef80 con 0x55985a9c9a30 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681ae1e0 con 0x55985a9ca710 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe471063740 10 client.0 _ll_register_callbacks cb 0x7fe45c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52082/0 (socket says 172.21.15.49:52082) 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 -- 172.21.15.49:0/470554405 learned_addr learned my addr 172.21.15.49:0/470554405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe4677fe640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe467fff640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 -- 172.21.15.49:0/470554405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 msgr2=0x55985a9cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 -- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 msgr2=0x55985a9db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 -- 172.21.15.49:0/470554405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985a9cca70 con 0x55985a9d2da0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe46ca1b640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe46000f5c0 tx=0x7fe460005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a24e81f11aa020 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe4477fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.745+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46002b040 con 0x55985a9d2da0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe460011630 con 0x55985a9d2da0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe45801f7d0 con 0x55985a9d2da0 2024-03-16T08:21:25.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe46003e050 con 0x55985a9d2da0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 10 client.16677 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55985a9cdd70 con 0x55985a9d2da0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45801f7d0 con 0x55985a9d2da0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe460030520 con 0x55985a9d2da0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 10 client.16677 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 20 client.16677 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985aa4a3c0 0x55985aa4c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 4 client.16677 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55985a9e55d0 con 0x55985aa4a3c0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.746+0000 7fe467fff640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985aa4a3c0 0x55985aa4c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.747+0000 7fe467fff640 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985aa4a3c0 0x55985aa4c7e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b414bcec2aae2620 server_cookie=223e8813e5b5f68f in_seq=0 out_seq=0 2024-03-16T08:21:25.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.747+0000 7fe4477fe640 10 client.16677 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe4477fe640 1 -- 172.21.15.49:0/470554405 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55985a9e55d0 con 0x55985aa4a3c0 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe4477fe640 10 client.16677 handle_command_reply: tid=0 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 client.16677 shutdown 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 msgr2=0x55985a9cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe46000f5c0 tx=0x7fe460005990 comp rx=0 tx=0).stop 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 shutdown_connections 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55985aa4a3c0 0x55985aa4c7e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55985a9d2da0 0x55985a9cc530 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe4477fe640 0 client.16677 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55985a9ca710 0x55985a9cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 --2- 172.21.15.49:0/470554405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55985a9c9a30 0x55985a9db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 >> 172.21.15.49:0/470554405 conn(0x55985a990c50 msgr2=0x55985a9d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 shutdown_connections 2024-03-16T08:21:25.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 1 -- 172.21.15.49:0/470554405 wait complete. 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.748+0000 7fe471063740 20 client.16677 trim_cache size 0 max 16384 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "b12cb63b-f457-41a4-b4d3-a0a01356c463", 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:21:25.756 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 msgr2=0x7fe4500a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 0x7fe4500a4fd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe4600099e0 tx=0x7fe46001e000 comp rx=0 tx=0).stop 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 msgr2=0x7fe468074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009230 tx=0x7fe45c02cb30 comp rx=0 tx=0).stop 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 shutdown_connections 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe4500a2b10 0x7fe4500a4fd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468107790 0x7fe468074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468102140 0x7fe468074020 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 --2- 172.21.15.49:0/3518235789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe468101780 0x7fe468073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 >> 172.21.15.49:0/3518235789 conn(0x7fe4680fd210 msgr2=0x7fe4680fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 shutdown_connections 2024-03-16T08:21:25.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.749+0000 7fe46e47e640 1 -- 172.21.15.49:0/3518235789 wait complete. 2024-03-16T08:21:25.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:21:25.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 -- 172.21.15.49:0/3312222833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100f20 msgr2=0x7f179810d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3312222833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100f20 0x7f179810d3e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1788009900 tx=0x7f178802bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 -- 172.21.15.49:0/3312222833 shutdown_connections 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3312222833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100f20 0x7f179810d3e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3312222833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100560 0x7f17981009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3312222833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 0x7f1798106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.870+0000 7f179dd9d640 1 -- 172.21.15.49:0/3312222833 >> 172.21.15.49:0/3312222833 conn(0x7f17980fc030 msgr2=0x7f17980fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- 172.21.15.49:0/3312222833 shutdown_connections 2024-03-16T08:21:25.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- 172.21.15.49:0/3312222833 wait complete. 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 Processor -- start 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- start start 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100f20 0x7f1798198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 0x7f179819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17981108e0 con 0x7f1798100560 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110760 con 0x7f1798100f20 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179dd9d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110a60 con 0x7f1798106620 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52102/0 (socket says 172.21.15.49:52102) 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179cd9b640 1 -- 172.21.15.49:0/3005403572 learned_addr learned my addr 172.21.15.49:0/3005403572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f1797fff640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100f20 0x7f1798198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.871+0000 7f179d59c640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 0x7f179819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 -- 172.21.15.49:0/3005403572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 msgr2=0x7f179819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 0x7f179819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 -- 172.21.15.49:0/3005403572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100f20 msgr2=0x7f1798198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100f20 0x7f1798198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 -- 172.21.15.49:0/3005403572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179819cdf0 con 0x7f1798100560 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1790009d50 tx=0x7f179000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f1795ffb640 1 -- 172.21.15.49:0/3005403572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790003b70 con 0x7f1798100560 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f1795ffb640 1 -- 172.21.15.49:0/3005403572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790003d10 con 0x7f1798100560 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179819d0e0 con 0x7f1798100560 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f1795ffb640 1 -- 172.21.15.49:0/3005403572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790010a20 con 0x7f1798100560 2024-03-16T08:21:25.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.872+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981a4970 con 0x7f1798100560 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.874+0000 7f1795ffb640 1 -- 172.21.15.49:0/3005403572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f1790003ec0 con 0x7f1798100560 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 Processor -- start 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 -- start start 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f1795ffb640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 0x7f17800a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebf32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebf3820 0x56202ebf5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56202eb4e500 con 0x56202ebea320 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56202eaf6be0 con 0x56202ebeac40 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f17a1bef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56202eb4a5a0 con 0x56202ebf3820 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179d59c640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 0x7f17800a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f1795ffb640 1 -- 172.21.15.49:0/3005403572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f17900cf4a0 con 0x7f1798100560 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52114/0 (socket says 172.21.15.49:52114) 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 -- 172.21.15.49:0/1525375268 learned_addr learned my addr 172.21.15.49:0/1525375268 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f1797fff640 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebf32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179d59c640 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebf3820 0x56202ebf5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 -- 172.21.15.49:0/1525375268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebf3820 msgr2=0x56202ebf5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebf3820 0x56202ebf5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 -- 172.21.15.49:0/1525375268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 msgr2=0x56202ebf32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebf32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.875+0000 7f179cd9b640 1 -- 172.21.15.49:0/1525375268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202eb0a4b0 con 0x56202ebea320 2024-03-16T08:21:25.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f179cd9b640 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1790000f80 tx=0x7f179009b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca50f8e7eaca0b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f179d59c640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 0x7f17800a4e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1788005ab0 tx=0x7f1788005d50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f177effd640 1 -- 172.21.15.49:0/1525375268 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790009820 con 0x56202ebea320 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f177effd640 1 -- 172.21.15.49:0/1525375268 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179009c250 con 0x56202ebea320 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f177effd640 1 -- 172.21.15.49:0/1525375268 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790004620 con 0x56202ebea320 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 -- 172.21.15.49:0/1525375268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 msgr2=0x56202ebea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1790000f80 tx=0x7f179009b4b0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 -- 172.21.15.49:0/1525375268 shutdown_connections 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebf3820 0x56202ebf5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebf32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 --2- 172.21.15.49:0/1525375268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebea700 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 -- 172.21.15.49:0/1525375268 >> 172.21.15.49:0/1525375268 conn(0x56202ebe4a50 msgr2=0x56202ebe4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 -- 172.21.15.49:0/1525375268 shutdown_connections 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.876+0000 7f17a1bef740 1 -- 172.21.15.49:0/1525375268 wait complete. 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 -- start start 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 Processor -- start 2024-03-16T08:21:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 0x56202ebf0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebf0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56202ebf65b0 con 0x56202ebf3820 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56202eb0a160 con 0x56202ebea320 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56202ebf6730 con 0x56202ebeac40 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f17a1bef740 10 client.0 _ll_register_callbacks cb 0x56202eaee150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f179d59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 0x56202ebf0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f179d59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52128/0 (socket says 172.21.15.49:52128) 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f1797fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebf0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.877+0000 7f179d59c640 1 -- 172.21.15.49:0/2720863176 learned_addr learned my addr 172.21.15.49:0/2720863176 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 -- 172.21.15.49:0/2720863176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 msgr2=0x56202ebf0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebf0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 -- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 msgr2=0x56202ebf0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 0x56202ebf0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 -- 172.21.15.49:0/2720863176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202e92c7d0 con 0x56202ebf3820 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179cd9b640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 0x56202ebf0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f179d59c640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1788035000 tx=0x7f178802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b276fe587ab3388 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004f10 con 0x56202ebf3820 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f17880050b0 con 0x56202ebf3820 2024-03-16T08:21:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179009c250 con 0x56202ebf3820 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f178805b050 con 0x56202ebf3820 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f17a1bef740 10 client.16689 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56202e8623f0 con 0x56202ebf3820 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179009c250 con 0x56202ebf3820 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.878+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f178804d450 con 0x56202ebf3820 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 10 client.16689 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 20 client.16689 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec03750 0x56202ec05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 4 client.16689 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202eb5ba40 con 0x56202ec03750 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f179cd9b640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec03750 0x56202ec05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f179cd9b640 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec03750 0x56202ec05bb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b116d73060bac85c server_cookie=9711f3a1dadeb955 in_seq=0 out_seq=0 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.879+0000 7f1777fff640 10 client.16689 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f1777fff640 1 -- 172.21.15.49:0/2720863176 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56202eb5ba40 con 0x56202ec03750 2024-03-16T08:21:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f1777fff640 10 client.16689 handle_command_reply: tid=0 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 client.16689 shutdown 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 msgr2=0x56202ebf12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1788035000 tx=0x7f178802bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 shutdown_connections 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec03750 0x56202ec05bb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebf12c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebf0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 --2- 172.21.15.49:0/2720863176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebea320 0x56202ebf0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f1777fff640 0 client.16689 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 >> 172.21.15.49:0/2720863176 conn(0x56202ebe4a50 msgr2=0x56202ebe67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 shutdown_connections 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.880+0000 7f17a1bef740 1 -- 172.21.15.49:0/2720863176 wait complete. 2024-03-16T08:21:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.881+0000 7f17a1bef740 20 client.16689 trim_cache size 0 max 16384 2024-03-16T08:21:25.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 Processor -- start 2024-03-16T08:21:25.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 -- start start 2024-03-16T08:21:25.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebf3820 0x56202ebe6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17981afa30 con 0x56202ebea320 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17981ad300 con 0x56202ebf3820 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f17a1bef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17981ab130 con 0x56202ebeac40 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52132/0 (socket says 172.21.15.49:52132) 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179cd9b640 1 -- 172.21.15.49:0/3137138909 learned_addr learned my addr 172.21.15.49:0/3137138909 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f1797fff640 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179d59c640 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebf3820 0x56202ebe6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179cd9b640 1 -- 172.21.15.49:0/3137138909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 msgr2=0x56202ebff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.898+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f179cd9b640 1 -- 172.21.15.49:0/3137138909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebf3820 msgr2=0x56202ebe6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebf3820 0x56202ebe6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f179cd9b640 1 -- 172.21.15.49:0/3137138909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202e8220f0 con 0x56202ebea320 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1790000f80 tx=0x7f1790005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1c50d7a78ea86c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f177e7fc640 1 -- 172.21.15.49:0/3137138909 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790009820 con 0x56202ebea320 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f177e7fc640 1 -- 172.21.15.49:0/3137138909 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179009c250 con 0x56202ebea320 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f177e7fc640 1 -- 172.21.15.49:0/3137138909 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900ccec0 con 0x56202ebea320 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 -- 172.21.15.49:0/3137138909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 msgr2=0x56202ebf2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1790000f80 tx=0x7f1790005f30 comp rx=0 tx=0).stop 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 -- 172.21.15.49:0/3137138909 shutdown_connections 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebf3820 0x56202ebe6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebeac40 0x56202ebff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3137138909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebea320 0x56202ebf2f70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.899+0000 7f17a1bef740 1 -- 172.21.15.49:0/3137138909 >> 172.21.15.49:0/3137138909 conn(0x56202ebe4a50 msgr2=0x56202ebe5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- 172.21.15.49:0/3137138909 shutdown_connections 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- 172.21.15.49:0/3137138909 wait complete. 2024-03-16T08:21:25.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- start start 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 Processor -- start 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 0x56202ebfbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17981ab7c0 con 0x56202ebf3820 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17981acc30 con 0x56202ebeac40 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17981abe90 con 0x56202ebea320 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f17a1bef740 10 client.0 _ll_register_callbacks cb 0x7f17900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f179d59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 0x56202ebfbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f1797fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f179d59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52148/0 (socket says 172.21.15.49:52148) 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f179cd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 0x56202ebfbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:57760/0 (socket says 172.21.15.49:57760) 2024-03-16T08:21:25.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.900+0000 7f179d59c640 1 -- 172.21.15.49:0/3423753335 learned_addr learned my addr 172.21.15.49:0/3423753335 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 -- 172.21.15.49:0/3423753335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 msgr2=0x56202ebfbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 0x56202ebfbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 -- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 msgr2=0x56202ebed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 -- 172.21.15.49:0/3423753335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202eb159d0 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1797fff640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f179d59c640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1788009900 tx=0x7f1788009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826e56502a123475 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1777fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178804d660 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1788004f10 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179009c250 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f178806d050 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f17a1bef740 10 client.16695 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56202ec69820 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.901+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179009c250 con 0x56202ebf3820 2024-03-16T08:21:25.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f17880039d0 con 0x56202ebf3820 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 10 client.16695 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 20 client.16695 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec69ff0 0x56202ec6c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 4 client.16695 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56202eb55d90 con 0x56202ec69ff0 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec69ff0 0x56202ec6c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f1777fff640 10 client.16695 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f179cd9b640 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec69ff0 0x56202ec6c410 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f6afebdf57465 server_cookie=8b5f77aca3d837de in_seq=0 out_seq=0 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f1777fff640 1 -- 172.21.15.49:0/3423753335 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56202eb55d90 con 0x56202ec69ff0 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f1777fff640 10 client.16695 handle_command_reply: tid=0 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 client.16695 shutdown 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 msgr2=0x56202ebed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1788009900 tx=0x7f1788009590 comp rx=0 tx=0).stop 2024-03-16T08:21:25.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 shutdown_connections 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56202ec69ff0 0x56202ec6c410 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56202ebf3820 0x56202ebed650 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.902+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56202ebeac40 0x56202ebed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f1777fff640 0 client.16695 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f17a1bef740 1 --2- 172.21.15.49:0/3423753335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56202ebea320 0x56202ebfbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 >> 172.21.15.49:0/3423753335 conn(0x56202ebe4a50 msgr2=0x56202ebf1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 shutdown_connections 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f17a1bef740 1 -- 172.21.15.49:0/3423753335 wait complete. 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f17a1bef740 20 client.16695 trim_cache size 0 max 16384 2024-03-16T08:21:25.910 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "b12cb63b-f457-41a4-b4d3-a0a01356c463": { 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "b12cb63b-f457-41a4-b4d3-a0a01356c463", 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:21:25.911 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 msgr2=0x7f17800a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 0x7f17800a4e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1788005ab0 tx=0x7f1788005d50 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 msgr2=0x7f1798197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1790009d50 tx=0x7f179000c6a0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 shutdown_connections 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17800a2970 0x7f17800a4e30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1798106620 0x7f179819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1798100f20 0x7f1798198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 --2- 172.21.15.49:0/3005403572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1798100560 0x7f1798197e00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 >> 172.21.15.49:0/3005403572 conn(0x7f17980fc030 msgr2=0x7f17980fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:25.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 shutdown_connections 2024-03-16T08:21:25.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:25.903+0000 7f179dd9d640 1 -- 172.21.15.49:0/3005403572 wait complete. 2024-03-16T08:21:25.915 INFO:tasks.cephfs.filesystem:scrub status for tag:b12cb63b-f457-41a4-b4d3-a0a01356c463 - {'path': '/', 'tag': 'b12cb63b-f457-41a4-b4d3-a0a01356c463', 'options': 'recursive,force'} 2024-03-16T08:21:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:26 smithi154 ceph-mon[36471]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:26 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:21:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:26 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:26 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:21:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:26 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:26 smithi049 ceph-mon[31942]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:26 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:21:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:26 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:26 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:21:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:26 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:26 smithi129 ceph-mon[36561]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:26.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:26 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:21:26.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:26 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:26.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:26 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:21:26.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:26 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:27 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:21:27.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:27 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:21:27.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:27 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:21:28.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:28 smithi154 ceph-mon[36471]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:28.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:28 smithi049 ceph-mon[31942]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:28 smithi129 ceph-mon[36561]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:30.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:30 smithi154 ceph-mon[36471]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:30.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:30 smithi049 ceph-mon[31942]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:30 smithi129 ceph-mon[36561]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:32.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:32 smithi154 ceph-mon[36471]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:32 smithi049 ceph-mon[31942]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:32.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:32 smithi129 ceph-mon[36561]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:34.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:34 smithi154 ceph-mon[36471]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:34.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:34 smithi049 ceph-mon[31942]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:34.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:34 smithi129 ceph-mon[36561]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:36.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:36 smithi154 ceph-mon[36471]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:36 smithi049 ceph-mon[31942]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:36.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:36 smithi129 ceph-mon[36561]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:38.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:38 smithi154 ceph-mon[36471]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:38.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:38 smithi049 ceph-mon[31942]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:38 smithi129 ceph-mon[36561]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:21:40.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:40 smithi154 ceph-mon[36471]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:40.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:40 smithi049 ceph-mon[31942]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:40.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:40 smithi129 ceph-mon[36561]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:21:42.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:42 smithi154 ceph-mon[36471]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:42.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:42 smithi049 ceph-mon[31942]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:42.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:42 smithi129 ceph-mon[36561]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:44.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:44 smithi154 ceph-mon[36471]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:44.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:44 smithi049 ceph-mon[31942]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:44.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:44 smithi129 ceph-mon[36561]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:46.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:46 smithi154 ceph-mon[36471]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:46 smithi049 ceph-mon[31942]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:46.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:46 smithi129 ceph-mon[36561]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:48.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:48 smithi154 ceph-mon[36471]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:48.794 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:48 smithi049 ceph-mon[31942]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:48.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:48 smithi129 ceph-mon[36561]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:50.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:50 smithi154 ceph-mon[36471]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:50 smithi049 ceph-mon[31942]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:50 smithi129 ceph-mon[36561]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:52.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:52 smithi154 ceph-mon[36471]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:52 smithi049 ceph-mon[31942]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:52 smithi129 ceph-mon[36561]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:54.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:54 smithi154 ceph-mon[36471]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:54 smithi049 ceph-mon[31942]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:54.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:54 smithi129 ceph-mon[36561]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:55.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:21:55.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:21:56.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.039+0000 7f2165587640 1 -- 172.21.15.49:0/2746575942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2160103370 msgr2=0x7f216010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.039+0000 7f2165587640 1 --2- 172.21.15.49:0/2746575942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2160103370 0x7f216010b850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2154009900 tx=0x7f215402bcd0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 -- 172.21.15.49:0/2746575942 shutdown_connections 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 --2- 172.21.15.49:0/2746575942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f216010be20 0x7f216010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 --2- 172.21.15.49:0/2746575942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2160103370 0x7f216010b850 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 --2- 172.21.15.49:0/2746575942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160102a50 0x7f2160102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 -- 172.21.15.49:0/2746575942 >> 172.21.15.49:0/2746575942 conn(0x7f2160077840 msgr2=0x7f2160077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 -- 172.21.15.49:0/2746575942 shutdown_connections 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 -- 172.21.15.49:0/2746575942 wait complete. 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 Processor -- start 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.040+0000 7f2165587640 1 -- start start 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 0x7f2160198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160103370 0x7f2160198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111bf0 con 0x7f216010be20 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111a70 con 0x7f2160102a50 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f2165587640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111d70 con 0x7f2160103370 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60494/0 (socket says 172.21.15.49:60494) 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 0x7f2160198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 -- 172.21.15.49:0/164983127 learned_addr learned my addr 172.21.15.49:0/164983127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215e7fc640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160103370 0x7f2160198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 -- 172.21.15.49:0/164983127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160103370 msgr2=0x7f2160198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160103370 0x7f2160198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 -- 172.21.15.49:0/164983127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 msgr2=0x7f2160198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 0x7f2160198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 -- 172.21.15.49:0/164983127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216019d0f0 con 0x7f216010be20 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215effd640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 0x7f2160198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.041+0000 7f215f7fe640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f215800eb10 tx=0x7f215800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.042+0000 7f2147fff640 1 -- 172.21.15.49:0/164983127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158004b20 con 0x7f216010be20 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.042+0000 7f2147fff640 1 -- 172.21.15.49:0/164983127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158004cc0 con 0x7f216010be20 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.042+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216019d3e0 con 0x7f216010be20 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.042+0000 7f2147fff640 1 -- 172.21.15.49:0/164983127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158010760 con 0x7f216010be20 2024-03-16T08:21:56.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.042+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21601a4c70 con 0x7f216010be20 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.044+0000 7f2147fff640 1 -- 172.21.15.49:0/164983127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2158005020 con 0x7f216010be20 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.044+0000 7f2168173740 1 Processor -- start 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.044+0000 7f2168173740 1 -- start start 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.044+0000 7f2147fff640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 0x7f21480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f25c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2147fff640 1 -- 172.21.15.49:0/164983127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f21580cf320 con 0x7f216010be20 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215e7fc640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 0x7f21480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f25ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55740f1b7500 con 0x55740f253a70 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55740f15fbe0 con 0x55740f25c650 2024-03-16T08:21:56.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48614/0 (socket says 172.21.15.49:48614) 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 -- 172.21.15.49:0/4124584551 learned_addr learned my addr 172.21.15.49:0/4124584551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55740f1b35a0 con 0x55740f253150 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215e7fc640 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f25c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215f7fe640 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f25ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 -- 172.21.15.49:0/4124584551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 msgr2=0x55740f25ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f25ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 -- 172.21.15.49:0/4124584551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 msgr2=0x55740f25c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f25c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 -- 172.21.15.49:0/4124584551 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740f1734b0 con 0x55740f253150 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215e7fc640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 0x7f21480a4e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f215402efd0 tx=0x7f215402ca20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.045+0000 7f215effd640 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21500115c0 tx=0x7f21500119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d16317323b979e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f21457fa640 1 -- 172.21.15.49:0/4124584551 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e070 con 0x55740f253150 2024-03-16T08:21:56.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f21457fa640 1 -- 172.21.15.49:0/4124584551 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215000fa20 con 0x55740f253150 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f21457fa640 1 -- 172.21.15.49:0/4124584551 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150019af0 con 0x55740f253150 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 msgr2=0x55740f253530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21500115c0 tx=0x7f21500119c0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 shutdown_connections 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f25ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f25c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 --2- 172.21.15.49:0/4124584551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f253530 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 >> 172.21.15.49:0/4124584551 conn(0x55740f24d880 msgr2=0x55740f24dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 shutdown_connections 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.046+0000 7f2168173740 1 -- 172.21.15.49:0/4124584551 wait complete. 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 -- start start 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 Processor -- start 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 0x55740f259c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f25c650 0x55740f25a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55740f25f440 con 0x55740f253150 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55740f173160 con 0x55740f253a70 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55740f25f5c0 con 0x55740f25c650 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f2168173740 10 client.0 _ll_register_callbacks cb 0x55740f157150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 0x55740f259c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60510/0 (socket says 172.21.15.49:60510) 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 -- 172.21.15.49:0/622285367 learned_addr learned my addr 172.21.15.49:0/622285367 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215f7fe640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f25c650 0x55740f25a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 -- 172.21.15.49:0/622285367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f25c650 msgr2=0x55740f25a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f25c650 0x55740f25a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 -- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 msgr2=0x55740f259c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 0x55740f259c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215effd640 1 -- 172.21.15.49:0/622285367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55740ef957d0 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.047+0000 7f215e7fc640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 0x55740f259c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f215effd640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2150005dd0 tx=0x7f21500056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c00807d99f74c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e040 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f215000f600 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f215000fa20 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2150030050 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 10 client.16707 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55740eecb3f0 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215000fa20 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2150021070 con 0x55740f253150 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 10 client.16707 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 20 client.16707 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.048+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f26c610 0x55740f26ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.049+0000 7f2168173740 4 client.16707 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.049+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55740f1c4a40 con 0x55740f26c610 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.049+0000 7f215e7fc640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f26c610 0x55740f26ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.049+0000 7f215e7fc640 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f26c610 0x55740f26ea00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81a543b10b943ad server_cookie=77fa714e5661af81 in_seq=0 out_seq=0 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.049+0000 7f2142ff5640 10 client.16707 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2142ff5640 1 -- 172.21.15.49:0/622285367 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55740f1c4a40 con 0x55740f26c610 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2142ff5640 10 client.16707 handle_command_reply: tid=0 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 client.16707 shutdown 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 msgr2=0x55740f259710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2150005dd0 tx=0x7f21500056d0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 shutdown_connections 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f26c610 0x55740f26ea00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f25c650 0x55740f25a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f253a70 0x55740f259c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2142ff5640 0 client.16707 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 --2- 172.21.15.49:0/622285367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f259710 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 >> 172.21.15.49:0/622285367 conn(0x55740f24d880 msgr2=0x55740f24f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 shutdown_connections 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 1 -- 172.21.15.49:0/622285367 wait complete. 2024-03-16T08:21:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.050+0000 7f2168173740 20 client.16707 trim_cache size 0 max 16384 2024-03-16T08:21:56.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.067+0000 7f2168173740 1 Processor -- start 2024-03-16T08:21:56.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 -- start start 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f25be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f24dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21601afd20 con 0x55740f253a70 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21601ad5f0 con 0x55740f25c650 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2168173740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21601ab420 con 0x55740f253150 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60526/0 (socket says 172.21.15.49:60526) 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 -- 172.21.15.49:0/3569127337 learned_addr learned my addr 172.21.15.49:0/3569127337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215effd640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f25be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215f7fe640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f24dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 -- 172.21.15.49:0/3569127337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 msgr2=0x55740f25be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f25be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 -- 172.21.15.49:0/3569127337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 msgr2=0x55740f24dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f24dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 -- 172.21.15.49:0/3569127337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740ee8b0f0 con 0x55740f253a70 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215f7fe640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f24dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f215e7fc640 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21540095c0 tx=0x7f2154046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82983a3125ca36aa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.068+0000 7f2144ff9640 1 -- 172.21.15.49:0/3569127337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154004c90 con 0x55740f253a70 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2144ff9640 1 -- 172.21.15.49:0/3569127337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21540044d0 con 0x55740f253a70 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2144ff9640 1 -- 172.21.15.49:0/3569127337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154005170 con 0x55740f253a70 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- 172.21.15.49:0/3569127337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 msgr2=0x55740f264bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21540095c0 tx=0x7f2154046000 comp rx=0 tx=0).stop 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- 172.21.15.49:0/3569127337 shutdown_connections 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f24dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253a70 0x55740f264bf0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 --2- 172.21.15.49:0/3569127337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253150 0x55740f25be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- 172.21.15.49:0/3569127337 >> 172.21.15.49:0/3569127337 conn(0x55740f24d880 msgr2=0x55740f24c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- 172.21.15.49:0/3569127337 shutdown_connections 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- 172.21.15.49:0/3569127337 wait complete. 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 -- start start 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.069+0000 7f2168173740 1 Processor -- start 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 0x55740f273fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f2cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21601abab0 con 0x55740f253150 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21601acf20 con 0x55740f25c650 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21601ac180 con 0x55740f253a70 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 10 client.0 _ll_register_callbacks cb 0x7f215001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60540/0 (socket says 172.21.15.49:60540) 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 -- 172.21.15.49:0/873090942 learned_addr learned my addr 172.21.15.49:0/873090942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215e7fc640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 0x55740f273fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215f7fe640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f2cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 -- 172.21.15.49:0/873090942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 msgr2=0x55740f273fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 0x55740f273fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 -- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 msgr2=0x55740f2cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f2cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 -- 172.21.15.49:0/873090942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55740f17e9d0 con 0x55740f253150 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215e7fc640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 0x55740f273fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215f7fe640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f2cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f215effd640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21500057c0 tx=0x7f2150009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc615f4ee7f7975 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2142ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e040 con 0x55740f253150 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f21500050d0 con 0x55740f253150 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.070+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21540044d0 con 0x55740f253150 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f215000fa10 con 0x55740f253150 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 10 client.16713 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55740f2d66e0 con 0x55740f253150 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21540044d0 con 0x55740f253150 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2150021050 con 0x55740f253150 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 10 client.16713 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 20 client.16713 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f2d6cf0 0x55740f2d9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 4 client.16713 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55740f1bed90 con 0x55740f2d6cf0 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.071+0000 7f215e7fc640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f2d6cf0 0x55740f2d9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2142ff5640 10 client.16713 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f215e7fc640 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f2d6cf0 0x55740f2d9110 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e859ae1e8e09b0c9 server_cookie=32fb78b6eb77417e in_seq=0 out_seq=0 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2142ff5640 1 -- 172.21.15.49:0/873090942 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55740f1bed90 con 0x55740f2d6cf0 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2142ff5640 10 client.16713 handle_command_reply: tid=0 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 client.16713 shutdown 2024-03-16T08:21:56.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 msgr2=0x55740f273a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21500057c0 tx=0x7f2150009fd0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 shutdown_connections 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55740f2d6cf0 0x55740f2d9110 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740f25c650 0x55740f2cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55740f253a70 0x55740f273fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2142ff5640 0 client.16713 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 --2- 172.21.15.49:0/873090942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55740f253150 0x55740f273a80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 >> 172.21.15.49:0/873090942 conn(0x55740f24d880 msgr2=0x55740f24c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 shutdown_connections 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 1 -- 172.21.15.49:0/873090942 wait complete. 2024-03-16T08:21:56.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2168173740 20 client.16713 trim_cache size 0 max 16384 2024-03-16T08:21:56.081 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:56.081 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:21:56.081 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:21:56.081 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 msgr2=0x7f21480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 0x7f21480a4e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f215402efd0 tx=0x7f215402ca20 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 msgr2=0x7f216019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.072+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f215800eb10 tx=0x7f215800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 shutdown_connections 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f21480a2970 0x7f21480a4e30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f216010be20 0x7f216019c9c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2160103370 0x7f2160198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 --2- 172.21.15.49:0/164983127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2160102a50 0x7f2160198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 >> 172.21.15.49:0/164983127 conn(0x7f2160077840 msgr2=0x7f21600fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 shutdown_connections 2024-03-16T08:21:56.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.073+0000 7f2165587640 1 -- 172.21.15.49:0/164983127 wait complete. 2024-03-16T08:21:56.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:21:56.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:21:56.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 -- 172.21.15.49:0/484637023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 msgr2=0x7f8e5c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/484637023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c102e30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 -- 172.21.15.49:0/484637023 shutdown_connections 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/484637023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 0x7f8e5c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/484637023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 0x7f8e5c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/484637023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c102e30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 -- 172.21.15.49:0/484637023 >> 172.21.15.49:0/484637023 conn(0x7f8e5c0777e0 msgr2=0x7f8e5c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 -- 172.21.15.49:0/484637023 shutdown_connections 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 -- 172.21.15.49:0/484637023 wait complete. 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.192+0000 7f8e61bb9640 1 Processor -- start 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 -- start start 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 0x7f8e5c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 0x7f8e5c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c111c20 con 0x7f8e5c102a50 2024-03-16T08:21:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c111aa0 con 0x7f8e5c103370 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e61bb9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c111da0 con 0x7f8e5c10be20 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60552/0 (socket says 172.21.15.49:60552) 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 -- 172.21.15.49:0/2386088054 learned_addr learned my addr 172.21.15.49:0/2386088054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5affd640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 0x7f8e5c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5bfff640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 0x7f8e5c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 -- 172.21.15.49:0/2386088054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 msgr2=0x7f8e5c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 0x7f8e5c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 -- 172.21.15.49:0/2386088054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 msgr2=0x7f8e5c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 0x7f8e5c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 -- 172.21.15.49:0/2386088054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c19d0a0 con 0x7f8e5c102a50 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e5b7fe640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e4402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44041070 con 0x7f8e5c102a50 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.194+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c19d330 con 0x7f8e5c102a50 2024-03-16T08:21:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.193+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e44004450 con 0x7f8e5c102a50 2024-03-16T08:21:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.194+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44003d00 con 0x7f8e5c102a50 2024-03-16T08:21:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.194+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c1a4c20 con 0x7f8e5c102a50 2024-03-16T08:21:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.194+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5c0685d0 con 0x7f8e5c102a50 2024-03-16T08:21:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.196+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8e44044070 con 0x7f8e5c102a50 2024-03-16T08:21:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.196+0000 7f8e58ff9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 0x7f8e480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.196+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8e440f07d0 con 0x7f8e5c102a50 2024-03-16T08:21:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.196+0000 7f8e5affd640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 0x7f8e480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.197+0000 7f8e5affd640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 0x7f8e480a4e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c10db40 tx=0x7f8e4c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.201+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e440bd690 con 0x7f8e5c102a50 2024-03-16T08:21:56.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.423+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5c106f10 con 0x7f8e5c102a50 2024-03-16T08:21:56.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.424+0000 7f8e58ff9640 1 -- 172.21.15.49:0/2386088054 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f8e440bcf90 con 0x7f8e5c102a50 2024-03-16T08:21:56.425 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:21:56.425 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:21:56.426 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:21:56.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 msgr2=0x7f8e480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 0x7f8e480a4e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c10db40 tx=0x7f8e4c005f90 comp rx=0 tx=0).stop 2024-03-16T08:21:56.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 msgr2=0x7f8e5c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e4402cb30 comp rx=0 tx=0).stop 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 shutdown_connections 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8e480a2970 0x7f8e480a4e30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e5c10be20 0x7f8e5c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e5c103370 0x7f8e5c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 --2- 172.21.15.49:0/2386088054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e5c102a50 0x7f8e5c198100 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 >> 172.21.15.49:0/2386088054 conn(0x7f8e5c0777e0 msgr2=0x7f8e5c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 shutdown_connections 2024-03-16T08:21:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.426+0000 7f8e61bb9640 1 -- 172.21.15.49:0/2386088054 wait complete. 2024-03-16T08:21:56.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:21:56.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 -- 172.21.15.49:0/1822043699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c0693a0 msgr2=0x7f4b4c10e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1822043699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c10e640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009cd0 tx=0x7f4b4402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 -- 172.21.15.49:0/1822043699 shutdown_connections 2024-03-16T08:21:56.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1822043699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c10e640 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1822043699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1822043699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c107830 0x7f4b4c107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 -- 172.21.15.49:0/1822043699 >> 172.21.15.49:0/1822043699 conn(0x7f4b4c0777e0 msgr2=0x7f4b4c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 -- 172.21.15.49:0/1822043699 shutdown_connections 2024-03-16T08:21:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 -- 172.21.15.49:0/1822043699 wait complete. 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.548+0000 7f4b50a59640 1 Processor -- start 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 -- start start 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111b80 con 0x7f4b4c107830 2024-03-16T08:21:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111a00 con 0x7f4b4c0689e0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b50a59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111d00 con 0x7f4b4c0693a0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60576/0 (socket says 172.21.15.49:60576) 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 -- 172.21.15.49:0/1557171537 learned_addr learned my addr 172.21.15.49:0/1557171537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 -- 172.21.15.49:0/1557171537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c0693a0 msgr2=0x7f4b4c100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 -- 172.21.15.49:0/1557171537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 msgr2=0x7f4b4c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 -- 172.21.15.49:0/1557171537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c1a64e0 con 0x7f4b4c107830 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.549+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004af0 tx=0x7f4b44032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.550+0000 7f4b437fe640 1 -- 172.21.15.49:0/1557171537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44042020 con 0x7f4b4c107830 2024-03-16T08:21:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.550+0000 7f4b437fe640 1 -- 172.21.15.49:0/1557171537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b44032e80 con 0x7f4b4c107830 2024-03-16T08:21:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.550+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c1a6770 con 0x7f4b4c107830 2024-03-16T08:21:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.550+0000 7f4b437fe640 1 -- 172.21.15.49:0/1557171537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44039040 con 0x7f4b4c107830 2024-03-16T08:21:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.550+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1a6c60 con 0x7f4b4c107830 2024-03-16T08:21:56.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.552+0000 7f4b437fe640 1 -- 172.21.15.49:0/1557171537 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4b44045020 con 0x7f4b4c107830 2024-03-16T08:21:56.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.552+0000 7f4b53642740 1 Processor -- start 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.552+0000 7f4b53642740 1 -- start start 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f332c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b437fe640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 0x7f4b380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f33e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55746f296490 con 0x55746f332830 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55746f23ec90 con 0x55746f33bd30 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b53642740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55746f292530 con 0x55746f333150 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f332c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48690/0 (socket says 172.21.15.49:48690) 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1164800917 learned_addr learned my addr 172.21.15.49:0/1164800917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f332c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60588/0 (socket says 172.21.15.49:60588) 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b437fe640 1 -- 172.21.15.49:0/1557171537 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4b440f0870 con 0x7f4b4c107830 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f33e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 0x7f4b380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1164800917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 msgr2=0x55746f33e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f33e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1164800917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 msgr2=0x55746f332c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f332c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1164800917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746f2f9500 con 0x55746f333150 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.553+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0114c0 tx=0x7f4b3c011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c50f4231c5728a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 0x7f4b380a4e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b340058c0 tx=0x7f4b34005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b40ff9640 1 -- 172.21.15.49:0/1164800917 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c004020 con 0x55746f333150 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b40ff9640 1 -- 172.21.15.49:0/1164800917 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c0041c0 con 0x55746f333150 2024-03-16T08:21:56.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b40ff9640 1 -- 172.21.15.49:0/1164800917 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00f8f0 con 0x55746f333150 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 -- 172.21.15.49:0/1164800917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 msgr2=0x55746f33b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0114c0 tx=0x7f4b3c011980 comp rx=0 tx=0).stop 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 -- 172.21.15.49:0/1164800917 shutdown_connections 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f33e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f333150 0x55746f33b7f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 --2- 172.21.15.49:0/1164800917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f332c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.554+0000 7f4b53642740 1 -- 172.21.15.49:0/1164800917 >> 172.21.15.49:0/1164800917 conn(0x55746f32cf60 msgr2=0x55746f32d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- 172.21.15.49:0/1164800917 shutdown_connections 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- 172.21.15.49:0/1164800917 wait complete. 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- start start 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 Processor -- start 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f339310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f339850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55746f33eb20 con 0x55746f332830 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55746f252230 con 0x55746f333150 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55746f33eca0 con 0x55746f33bd30 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b53642740 10 client.0 _ll_register_callbacks cb 0x55746f28e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60600/0 (socket says 172.21.15.49:60600) 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b4a59c640 1 -- 172.21.15.49:0/3970214784 learned_addr learned my addr 172.21.15.49:0/3970214784 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b43fff640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f339310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.555+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f339850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 -- 172.21.15.49:0/3970214784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 msgr2=0x55746f339850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f339850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 -- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 msgr2=0x55746f339310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f339310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 -- 172.21.15.49:0/3970214784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746f261100 con 0x55746f332830 2024-03-16T08:21:56.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b43fff640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f339310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01b040 tx=0x7f4b3c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c08d2ca824c90 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c01d040 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4b3c016050 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3c0041c0 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c003c60 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b53642740 10 client.16731 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55746efaa3f0 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c0041c0 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.556+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b3c020030 con 0x55746f332830 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 10 client.16731 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 10 client.16731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 20 client.16731 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f34c000 0x55746f34e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 4 client.16731 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55746f2a39d0 con 0x55746f34c000 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b43fff640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f34c000 0x55746f34e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b31ffb640 10 client.16731 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.557+0000 7f4b43fff640 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f34c000 0x55746f34e3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d9fe54f8291b02c server_cookie=cc37df5e9f774ec8 in_seq=0 out_seq=0 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b31ffb640 1 -- 172.21.15.49:0/3970214784 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55746f2a39d0 con 0x55746f34c000 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b31ffb640 10 client.16731 handle_command_reply: tid=0 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 client.16731 shutdown 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 msgr2=0x55746f338dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01b040 tx=0x7f4b3c005cf0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 shutdown_connections 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f34c000 0x55746f34e3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f339850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b31ffb640 0 client.16731 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f339310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 --2- 172.21.15.49:0/3970214784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f338dd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 >> 172.21.15.49:0/3970214784 conn(0x55746f32cf60 msgr2=0x55746f32ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 shutdown_connections 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 1 -- 172.21.15.49:0/3970214784 wait complete. 2024-03-16T08:21:56.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.558+0000 7f4b53642740 20 client.16731 trim_cache size 0 max 16384 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 Processor -- start 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 -- start start 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f332830 0x55746f33b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f353c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1b1e90 con 0x55746f333150 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1af760 con 0x55746f33bd30 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.571+0000 7f4b53642740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1ad5b0 con 0x55746f332830 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60608/0 (socket says 172.21.15.49:60608) 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 -- 172.21.15.49:0/1168644568 learned_addr learned my addr 172.21.15.49:0/1168644568 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f332830 0x55746f33b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f353c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 -- 172.21.15.49:0/1168644568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f332830 msgr2=0x55746f33b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f332830 0x55746f33b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 -- 172.21.15.49:0/1168644568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 msgr2=0x55746f353c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f353c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 -- 172.21.15.49:0/1168644568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746f0747d0 con 0x55746f333150 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55746f338630 tx=0x7f4b34005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1fd512bf3f20b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b33fff640 1 -- 172.21.15.49:0/1168644568 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3402cb90 con 0x55746f333150 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b33fff640 1 -- 172.21.15.49:0/1168644568 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34005020 con 0x55746f333150 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.572+0000 7f4b33fff640 1 -- 172.21.15.49:0/1168644568 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b340049e0 con 0x55746f333150 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- 172.21.15.49:0/1168644568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 msgr2=0x55746f3478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55746f338630 tx=0x7f4b34005850 comp rx=0 tx=0).stop 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- 172.21.15.49:0/1168644568 shutdown_connections 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f33bd30 0x55746f353c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f333150 0x55746f3478c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- 172.21.15.49:0/1168644568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f332830 0x55746f33b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- 172.21.15.49:0/1168644568 >> 172.21.15.49:0/1168644568 conn(0x55746f32cf60 msgr2=0x55746f331fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- 172.21.15.49:0/1168644568 shutdown_connections 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- 172.21.15.49:0/1168644568 wait complete. 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- start start 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 Processor -- start 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f33eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f33f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1adc20 con 0x55746f332830 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1af090 con 0x55746f333150 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1ae2f0 con 0x55746f33bd30 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b53642740 10 client.0 _ll_register_callbacks cb 0x7f4b34009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.573+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60612/0 (socket says 172.21.15.49:60612) 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1673023327 learned_addr learned my addr 172.21.15.49:0/1673023327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4ad9d640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f33f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f33eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1673023327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 msgr2=0x55746f33f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f33f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 msgr2=0x55746f33eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f33eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 -- 172.21.15.49:0/1673023327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746f33fcb0 con 0x55746f332830 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b4a59c640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0049b0 tx=0x7f4b3c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e7626cbc429fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c039060 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f4b3c011250 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b34005020 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c01d040 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b53642740 10 client.16737 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55746f3b1380 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.574+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34005020 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b3c033030 con 0x55746f332830 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 10 client.16737 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 10 client.16737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 20 client.16737 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f3b1b00 0x55746f3b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 4 client.16737 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55746f29dd20 con 0x55746f3b1b00 2024-03-16T08:21:56.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f3b1b00 0x55746f3b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b31ffb640 10 client.16737 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b43fff640 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f3b1b00 0x55746f3b3f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=507f3507dce9c57b server_cookie=c89aa97ba1728e75 in_seq=0 out_seq=0 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b31ffb640 1 -- 172.21.15.49:0/1673023327 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55746f29dd20 con 0x55746f3b1b00 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b31ffb640 10 client.16737 handle_command_reply: tid=0 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 1 client.16737 shutdown 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 msgr2=0x55746f344260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.575+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0049b0 tx=0x7f4b3c004c60 comp rx=0 tx=0).stop 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 shutdown_connections 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55746f3b1b00 0x55746f3b3f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55746f33bd30 0x55746f33f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b31ffb640 0 client.16737 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55746f333150 0x55746f33eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 --2- 172.21.15.49:0/1673023327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746f332830 0x55746f344260 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 >> 172.21.15.49:0/1673023327 conn(0x55746f32cf60 msgr2=0x55746f337660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 shutdown_connections 2024-03-16T08:21:56.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 1 -- 172.21.15.49:0/1673023327 wait complete. 2024-03-16T08:21:56.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b53642740 20 client.16737 trim_cache size 0 max 16384 2024-03-16T08:21:56.585 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 msgr2=0x7f4b380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 0x7f4b380a4e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b340058c0 tx=0x7f4b34005f90 comp rx=0 tx=0).stop 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 msgr2=0x7f4b4c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004af0 tx=0x7f4b44032c90 comp rx=0 tx=0).stop 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 shutdown_connections 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b380a2970 0x7f4b380a4e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b4c107830 0x7f4b4c101120 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b4c0693a0 0x7f4b4c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 --2- 172.21.15.49:0/1557171537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b4c0689e0 0x7f4b4c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 >> 172.21.15.49:0/1557171537 conn(0x7f4b4c0777e0 msgr2=0x7f4b4c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 shutdown_connections 2024-03-16T08:21:56.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.576+0000 7f4b50a59640 1 -- 172.21.15.49:0/1557171537 wait complete. 2024-03-16T08:21:56.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:21:56.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.697+0000 7f1a48bdf640 1 -- 172.21.15.49:0/4122568212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 msgr2=0x7f1a4410e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.697+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/4122568212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a4410e290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009ca0 tx=0x7f1a3c02c180 comp rx=0 tx=0).stop 2024-03-16T08:21:56.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- 172.21.15.49:0/4122568212 shutdown_connections 2024-03-16T08:21:56.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/4122568212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a4410e290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/4122568212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 0x7f1a4410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/4122568212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a44102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- 172.21.15.49:0/4122568212 >> 172.21.15.49:0/4122568212 conn(0x7f1a44077840 msgr2=0x7f1a44077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- 172.21.15.49:0/4122568212 shutdown_connections 2024-03-16T08:21:56.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- 172.21.15.49:0/4122568212 wait complete. 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 Processor -- start 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- start start 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 0x7f1a44100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a44101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.698+0000 7f1a48bdf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44111bf0 con 0x7f1a44102a50 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a48bdf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44111a70 con 0x7f1a4410be20 2024-03-16T08:21:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a48bdf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44111d70 con 0x7f1a44103370 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60620/0 (socket says 172.21.15.49:60620) 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 -- 172.21.15.49:0/1691365645 learned_addr learned my addr 172.21.15.49:0/1691365645 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a44101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 0x7f1a44100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 -- 172.21.15.49:0/1691365645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 msgr2=0x7f1a44100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 0x7f1a44100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 -- 172.21.15.49:0/1691365645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 msgr2=0x7f1a44101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a44101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 -- 172.21.15.49:0/1691365645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441a6590 con 0x7f1a44102a50 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a44101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e9f0 tx=0x7f1a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a337fe640 1 -- 172.21.15.49:0/1691365645 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400cd80 con 0x7f1a44102a50 2024-03-16T08:21:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a441a6880 con 0x7f1a44102a50 2024-03-16T08:21:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a337fe640 1 -- 172.21.15.49:0/1691365645 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a34010910 con 0x7f1a44102a50 2024-03-16T08:21:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a337fe640 1 -- 172.21.15.49:0/1691365645 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34018990 con 0x7f1a44102a50 2024-03-16T08:21:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.699+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a6dd0 con 0x7f1a44102a50 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.702+0000 7f1a337fe640 1 -- 172.21.15.49:0/1691365645 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f1a34010ac0 con 0x7f1a44102a50 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.702+0000 7f1a4b7c8740 1 Processor -- start 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 -- start start 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3b830 0x56272ee3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56272ed9f490 con 0x56272ee44d30 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56272ed47c90 con 0x56272ee3b830 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56272ed9b530 con 0x56272ee3c150 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a337fe640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 0x7f1a2c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60628/0 (socket says 172.21.15.49:60628) 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 -- 172.21.15.49:0/2889295225 learned_addr learned my addr 172.21.15.49:0/2889295225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3b830 0x56272ee3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 0x7f1a2c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a337fe640 1 -- 172.21.15.49:0/1691365645 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1a340cf170 con 0x7f1a44102a50 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 -- 172.21.15.49:0/2889295225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 msgr2=0x56272ee447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 -- 172.21.15.49:0/2889295225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3b830 msgr2=0x56272ee3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3b830 0x56272ee3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.703+0000 7f1a41d9b640 1 -- 172.21.15.49:0/2889295225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272ee02500 con 0x56272ee44d30 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009c10 tx=0x7f1a38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8811a79e5f2918 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 0x7f1a2c0a4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c004af0 tx=0x7f1a3c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a30ff9640 1 -- 172.21.15.49:0/2889295225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38004230 con 0x56272ee44d30 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a30ff9640 1 -- 172.21.15.49:0/2889295225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a380043d0 con 0x56272ee44d30 2024-03-16T08:21:56.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a30ff9640 1 -- 172.21.15.49:0/2889295225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38004e00 con 0x56272ee44d30 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/2889295225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 msgr2=0x56272ee47120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.704+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009c10 tx=0x7f1a38011810 comp rx=0 tx=0).stop 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/2889295225 shutdown_connections 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee47120 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/2889295225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3b830 0x56272ee3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/2889295225 >> 172.21.15.49:0/2889295225 conn(0x56272ee35f60 msgr2=0x56272ee36340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/2889295225 shutdown_connections 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/2889295225 wait complete. 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- start start 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 Processor -- start 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56272ee47b20 con 0x56272ee3b830 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56272ed5b230 con 0x56272ee44d30 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56272ee47ca0 con 0x56272ee3c150 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4b7c8740 10 client.0 _ll_register_callbacks cb 0x56272ed97010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.705+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a41d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60644/0 (socket says 172.21.15.49:60644) 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 -- 172.21.15.49:0/4061933929 learned_addr learned my addr 172.21.15.49:0/4061933929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 -- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 msgr2=0x56272ee422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 -- 172.21.15.49:0/4061933929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 msgr2=0x56272ee42830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee42830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 -- 172.21.15.49:0/4061933929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272ed6a100 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4259c640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e9c0 tx=0x7f1a3409b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f328c3dfa607c8a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34026040 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1a3409cb50 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a380043d0 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a340ccd90 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4b7c8740 10 client.16749 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56272eab33f0 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.706+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a380043d0 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a3409ad20 con 0x56272ee3b830 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 10 client.16749 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 10 client.16749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 20 client.16749 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272ee550b0 0x56272ee574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 4 client.16749 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56272edac9d0 con 0x56272ee550b0 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272ee550b0 0x56272ee574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a29ffb640 10 client.16749 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.707+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272ee550b0 0x56272ee574c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e53db7295894aa21 server_cookie=e7136c693c1ad083 in_seq=0 out_seq=0 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a29ffb640 1 -- 172.21.15.49:0/4061933929 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56272edac9d0 con 0x56272ee550b0 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a29ffb640 10 client.16749 handle_command_reply: tid=0 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 client.16749 shutdown 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 msgr2=0x56272ee41db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e9c0 tx=0x7f1a3409b8d0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 shutdown_connections 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272ee550b0 0x56272ee574c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a29ffb640 0 client.16749 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/4061933929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee41db0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 >> 172.21.15.49:0/4061933929 conn(0x56272ee35f60 msgr2=0x56272ee37cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 shutdown_connections 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/4061933929 wait complete. 2024-03-16T08:21:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.708+0000 7f1a4b7c8740 20 client.16749 trim_cache size 0 max 16384 2024-03-16T08:21:56.728 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:56 smithi154 ceph-mon[36471]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:56.728 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:56 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2386088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.721+0000 7f1a4b7c8740 1 Processor -- start 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.721+0000 7f1a4b7c8740 1 -- start start 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee5cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441b1ee0 con 0x56272ee3b830 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441af7b0 con 0x56272ee44d30 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441ad5e0 con 0x56272ee3c150 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60650/0 (socket says 172.21.15.49:60650) 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 -- 172.21.15.49:0/1097109607 learned_addr learned my addr 172.21.15.49:0/1097109607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee5cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 -- 172.21.15.49:0/1097109607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 msgr2=0x56272ee508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 -- 172.21.15.49:0/1097109607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 msgr2=0x56272ee5cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee5cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 -- 172.21.15.49:0/1097109607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272eb7d7d0 con 0x56272ee3b830 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400cfd0 tx=0x7f1a34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c745fd426ad7999e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a2bfff640 1 -- 172.21.15.49:0/1097109607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340d1120 con 0x56272ee3b830 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a2bfff640 1 -- 172.21.15.49:0/1097109607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3409ad20 con 0x56272ee3b830 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.722+0000 7f1a2bfff640 1 -- 172.21.15.49:0/1097109607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34097030 con 0x56272ee3b830 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1097109607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 msgr2=0x56272ee444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400cfd0 tx=0x7f1a34000bc0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1097109607 shutdown_connections 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee44d30 0x56272ee5cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3c150 0x56272ee508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1097109607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee3b830 0x56272ee444e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1097109607 >> 172.21.15.49:0/1097109607 conn(0x56272ee35f60 msgr2=0x56272ee34750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1097109607 shutdown_connections 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1097109607 wait complete. 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 -- start start 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.723+0000 7f1a4b7c8740 1 Processor -- start 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 0x56272ee3eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 0x56272ee3eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441adc70 con 0x56272ee44d30 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441af0e0 con 0x56272ee3c150 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441ae340 con 0x56272ee3b830 2024-03-16T08:21:56.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 10 client.0 _ll_register_callbacks cb 0x7f1a3409b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60664/0 (socket says 172.21.15.49:60664) 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 -- 172.21.15.49:0/1078094282 learned_addr learned my addr 172.21.15.49:0/1078094282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 0x56272ee3eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 0x56272ee3eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 -- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 msgr2=0x56272ee3eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 0x56272ee3eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 -- 172.21.15.49:0/1078094282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 msgr2=0x56272ee3eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 0x56272ee3eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 -- 172.21.15.49:0/1078094282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272eeba520 con 0x56272ee44d30 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a41d9b640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 0x56272ee3eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 0x56272ee3eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a42d9d640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c036000 tx=0x7f1a3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47caa64973d43b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c055030 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3409ad20 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.724+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f1a3c032b70 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a3c05e030 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 10 client.16755 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56272eebb800 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3409ad20 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a3c004580 con 0x56272ee44d30 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 10 client.16755 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 20 client.16755 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272eebbfb0 0x56272eebe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 4 client.16755 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:56.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56272eda6d20 con 0x56272eebbfb0 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272eebbfb0 0x56272eebe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a29ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.725+0000 7f1a4259c640 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272eebbfb0 0x56272eebe3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5abbe20e5f2a014 server_cookie=8de8cec9271efe35 in_seq=0 out_seq=0 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a29ffb640 1 -- 172.21.15.49:0/1078094282 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56272eda6d20 con 0x56272eebbfb0 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a29ffb640 10 client.16755 handle_command_reply: tid=0 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 client.16755 shutdown 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 msgr2=0x56272ee3f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c036000 tx=0x7f1a3c0095c0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 shutdown_connections 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x56272eebbfb0 0x56272eebe3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56272ee44d30 0x56272ee3f530 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56272ee3c150 0x56272ee3eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a29ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:21:56.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 --2- 172.21.15.49:0/1078094282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56272ee3b830 0x56272ee3eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 >> 172.21.15.49:0/1078094282 conn(0x56272ee35f60 msgr2=0x56272ee42600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 shutdown_connections 2024-03-16T08:21:56.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 1 -- 172.21.15.49:0/1078094282 wait complete. 2024-03-16T08:21:56.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a4b7c8740 20 client.16755 trim_cache size 0 max 16384 2024-03-16T08:21:56.735 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 msgr2=0x7f1a2c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 0x7f1a2c0a4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c004af0 tx=0x7f1a3c005c50 comp rx=0 tx=0).stop 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 msgr2=0x7f1a441006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e9f0 tx=0x7f1a3400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 shutdown_connections 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a2c0a2970 0x7f1a2c0a4e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a4410be20 0x7f1a44101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a44103370 0x7f1a44100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 --2- 172.21.15.49:0/1691365645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a44102a50 0x7f1a441006a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.726+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 >> 172.21.15.49:0/1691365645 conn(0x7f1a44077840 msgr2=0x7f1a440fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.727+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 shutdown_connections 2024-03-16T08:21:56.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.727+0000 7f1a48bdf640 1 -- 172.21.15.49:0/1691365645 wait complete. 2024-03-16T08:21:56.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:56 smithi049 ceph-mon[31942]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:56.737 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:56 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2386088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:56.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 -- 172.21.15.49:0/1335214401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034100700 msgr2=0x7fc034100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 --2- 172.21.15.49:0/1335214401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034100700 0x7fc034100b80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc024009930 tx=0x7fc02402bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 -- 172.21.15.49:0/1335214401 shutdown_connections 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 --2- 172.21.15.49:0/1335214401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc03410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 --2- 172.21.15.49:0/1335214401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034100700 0x7fc034100b80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 --2- 172.21.15.49:0/1335214401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034106710 0x7fc034106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 -- 172.21.15.49:0/1335214401 >> 172.21.15.49:0/1335214401 conn(0x7fc034077840 msgr2=0x7fc034077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 -- 172.21.15.49:0/1335214401 shutdown_connections 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.847+0000 7fc03c597640 1 -- 172.21.15.49:0/1335214401 wait complete. 2024-03-16T08:21:56.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 Processor -- start 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 -- start start 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034100700 0x7fc0341980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc034198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc034110b70 con 0x7fc034106710 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341109f0 con 0x7fc0341010c0 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03c597640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc034110cf0 con 0x7fc034100700 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03ab34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03ab34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60692/0 (socket says 172.21.15.49:60692) 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc039b32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc034198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03ab34640 1 -- 172.21.15.49:0/1957319629 learned_addr learned my addr 172.21.15.49:0/1957319629 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.848+0000 7fc03a333640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034100700 0x7fc0341980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 -- 172.21.15.49:0/1957319629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034100700 msgr2=0x7fc0341980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034100700 0x7fc0341980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 -- 172.21.15.49:0/1957319629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 msgr2=0x7fc034198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc034198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 -- 172.21.15.49:0/1957319629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419d0e0 con 0x7fc034106710 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc039b32640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc034198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03ab34640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00efd0 tx=0x7fc02c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc02b7fe640 1 -- 172.21.15.49:0/1957319629 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c003b70 con 0x7fc034106710 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc02b7fe640 1 -- 172.21.15.49:0/1957319629 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc02c003d10 con 0x7fc034106710 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03419d3d0 con 0x7fc034106710 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc02b7fe640 1 -- 172.21.15.49:0/1957319629 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0109f0 con 0x7fc034106710 2024-03-16T08:21:56.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.849+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a4c60 con 0x7fc034106710 2024-03-16T08:21:56.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.851+0000 7fc02b7fe640 1 -- 172.21.15.49:0/1957319629 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc02c004020 con 0x7fc034106710 2024-03-16T08:21:56.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 Processor -- start 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 -- start start 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc857148c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc85721c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc85724530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc02b7fe640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 0x7fc0200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc85678490 con 0x55dc857144e0 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc85620c90 con 0x55dc85722140 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03f186740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc85674530 con 0x55dc85714e00 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc039b32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc857148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc039b32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc857148c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60704/0 (socket says 172.21.15.49:60704) 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc039b32640 1 -- 172.21.15.49:0/2067238758 learned_addr learned my addr 172.21.15.49:0/2067238758 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03a333640 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc85721c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc03ab34640 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc85724530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc039b32640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 0x7fc0200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.852+0000 7fc02b7fe640 1 -- 172.21.15.49:0/1957319629 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc02c0cf220 con 0x7fc034106710 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 -- 172.21.15.49:0/2067238758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 msgr2=0x55dc85724530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc85724530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 -- 172.21.15.49:0/2067238758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 msgr2=0x55dc857148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc857148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 -- 172.21.15.49:0/2067238758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc856db500 con 0x55dc85714e00 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc039b32640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 0x7fc0200a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc02402df90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc03a333640 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc85721c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc030009c00 tx=0x7fc030011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1ddf85a0069964 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc028bfe640 1 -- 172.21.15.49:0/2067238758 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030004050 con 0x55dc85714e00 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc028bfe640 1 -- 172.21.15.49:0/2067238758 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc030005500 con 0x55dc85714e00 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.853+0000 7fc028bfe640 1 -- 172.21.15.49:0/2067238758 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030004d90 con 0x55dc85714e00 2024-03-16T08:21:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- 172.21.15.49:0/2067238758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 msgr2=0x55dc85721c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc85721c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc030009c00 tx=0x7fc030011810 comp rx=0 tx=0).stop 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- 172.21.15.49:0/2067238758 shutdown_connections 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc85724530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc85721c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 --2- 172.21.15.49:0/2067238758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc857148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- 172.21.15.49:0/2067238758 >> 172.21.15.49:0/2067238758 conn(0x55dc8570ef60 msgr2=0x55dc8570f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- 172.21.15.49:0/2067238758 shutdown_connections 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- 172.21.15.49:0/2067238758 wait complete. 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 -- start start 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.854+0000 7fc03f186740 1 Processor -- start 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc857144e0 0x55dc8571f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85714e00 0x55dc8571f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc85724f80 con 0x55dc85722140 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc85634230 con 0x55dc85714e00 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc85725100 con 0x55dc857144e0 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 10 client.0 _ll_register_callbacks cb 0x55dc85670010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60718/0 (socket says 172.21.15.49:60718) 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 -- 172.21.15.49:0/2766284758 learned_addr learned my addr 172.21.15.49:0/2766284758 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03a333640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc857144e0 0x55dc8571f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc039b32640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85714e00 0x55dc8571f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 -- 172.21.15.49:0/2766284758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc857144e0 msgr2=0x55dc8571f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc857144e0 0x55dc8571f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 -- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85714e00 msgr2=0x55dc8571f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85714e00 0x55dc8571f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 -- 172.21.15.49:0/2766284758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc85643100 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03ab34640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cca20 tx=0x7fc02c00a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f112538a10afd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c009820 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc02c004620 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc030005500 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc02c09bfd0 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.855+0000 7fc03f186740 10 client.16767 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc8538c3f0 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc030005500 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc02c01f020 con 0x55dc85722140 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 10 client.16767 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 20 client.16767 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc857323c0 0x55dc857347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 4 client.16767 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc856859d0 con 0x55dc857323c0 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.856+0000 7fc03a333640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc857323c0 0x55dc857347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc01dffb640 10 client.16767 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03a333640 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc857323c0 0x55dc857347b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e191d97ed6c022 server_cookie=de52b6da177e1f33 in_seq=0 out_seq=0 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc01dffb640 1 -- 172.21.15.49:0/2766284758 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dc856859d0 con 0x55dc857323c0 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc01dffb640 10 client.16767 handle_command_reply: tid=0 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 client.16767 shutdown 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 msgr2=0x55dc8571fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cca20 tx=0x7fc02c00a9c0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 shutdown_connections 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc857323c0 0x55dc857347b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85722140 0x55dc8571fae0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85714e00 0x55dc8571f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 --2- 172.21.15.49:0/2766284758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc857144e0 0x55dc8571f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 >> 172.21.15.49:0/2766284758 conn(0x55dc8570ef60 msgr2=0x55dc85710cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc01dffb640 0 client.16767 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 shutdown_connections 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 1 -- 172.21.15.49:0/2766284758 wait complete. 2024-03-16T08:21:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.857+0000 7fc03f186740 20 client.16767 trim_cache size 0 max 16384 2024-03-16T08:21:56.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:56 smithi129 ceph-mon[36561]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:21:56.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:56 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2386088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 Processor -- start 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 -- start start 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc8572dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc8573a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ad9c0 con 0x55dc857144e0 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ab290 con 0x55dc85722140 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03f186740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341a90c0 con 0x55dc85714e00 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60720/0 (socket says 172.21.15.49:60720) 2024-03-16T08:21:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 -- 172.21.15.49:0/187644341 learned_addr learned my addr 172.21.15.49:0/187644341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03ab34640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc8573a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc039b32640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc8572dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 -- 172.21.15.49:0/187644341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 msgr2=0x55dc8572dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc8572dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 -- 172.21.15.49:0/187644341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 msgr2=0x55dc8573a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc8573a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 -- 172.21.15.49:0/187644341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc854567d0 con 0x55dc857144e0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc039b32640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc8572dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.871+0000 7fc03a333640 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc030005c70 tx=0x7fc030004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7b67f5f7e04322 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc01ffff640 1 -- 172.21.15.49:0/187644341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030033070 con 0x55dc857144e0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc01ffff640 1 -- 172.21.15.49:0/187644341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc03002ecb0 con 0x55dc857144e0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc01ffff640 1 -- 172.21.15.49:0/187644341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030004610 con 0x55dc857144e0 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- 172.21.15.49:0/187644341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 msgr2=0x55dc85721790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc030005c70 tx=0x7fc030004970 comp rx=0 tx=0).stop 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- 172.21.15.49:0/187644341 shutdown_connections 2024-03-16T08:21:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc85722140 0x55dc8573a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85714e00 0x55dc8572dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 --2- 172.21.15.49:0/187644341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc857144e0 0x55dc85721790 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- 172.21.15.49:0/187644341 >> 172.21.15.49:0/187644341 conn(0x55dc8570ef60 msgr2=0x55dc8570dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- 172.21.15.49:0/187644341 shutdown_connections 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- 172.21.15.49:0/187644341 wait complete. 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 -- start start 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.872+0000 7fc03f186740 1 Processor -- start 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc857144e0 0x55dc8572a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85722140 0x55dc857262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341a9750 con 0x55dc85714e00 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341aabc0 con 0x55dc857144e0 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341a9e20 con 0x55dc85722140 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03f186740 10 client.0 _ll_register_callbacks cb 0x7fc02c007ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60730/0 (socket says 172.21.15.49:60730) 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 -- 172.21.15.49:0/2637472942 learned_addr learned my addr 172.21.15.49:0/2637472942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03a333640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc857144e0 0x55dc8572a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc03ab34640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85722140 0x55dc857262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 -- 172.21.15.49:0/2637472942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85722140 msgr2=0x55dc857262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85722140 0x55dc857262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 -- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc857144e0 msgr2=0x55dc8572a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc857144e0 0x55dc8572a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 -- 172.21.15.49:0/2637472942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc85797660 con 0x55dc85714e00 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc039b32640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc024009900 tx=0x7fc024009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8f58ff3bfae7b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.873+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240053b0 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc024005550 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc03002ecb0 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc0240035f0 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 10 client.16773 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc85798850 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc03002ecb0 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc02405b6f0 con 0x55dc85714e00 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 10 client.16773 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 20 client.16773 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc85798fb0 0x55dc8579b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 4 client.16773 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc8567fd20 con 0x55dc85798fb0 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.874+0000 7fc03a333640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc85798fb0 0x55dc8579b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03a333640 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc85798fb0 0x55dc8579b3d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2b98e309d8b4ea server_cookie=dbf46d85a8b6acd8 in_seq=0 out_seq=0 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc01dffb640 10 client.16773 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc01dffb640 1 -- 172.21.15.49:0/2637472942 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc8567fd20 con 0x55dc85798fb0 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc01dffb640 10 client.16773 handle_command_reply: tid=0 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 client.16773 shutdown 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 msgr2=0x55dc85725ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc024009900 tx=0x7fc024009260 comp rx=0 tx=0).stop 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 shutdown_connections 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc85798fb0 0x55dc8579b3d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc85722140 0x55dc857262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc01dffb640 0 client.16773 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc85714e00 0x55dc85725ba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 --2- 172.21.15.49:0/2637472942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc857144e0 0x55dc8572a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 >> 172.21.15.49:0/2637472942 conn(0x55dc8570ef60 msgr2=0x55dc8571f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 shutdown_connections 2024-03-16T08:21:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 1 -- 172.21.15.49:0/2637472942 wait complete. 2024-03-16T08:21:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03f186740 20 client.16773 trim_cache size 0 max 16384 2024-03-16T08:21:56.884 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 msgr2=0x7fc0200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 0x7fc0200a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc02402df90 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 msgr2=0x7fc03419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.875+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00efd0 tx=0x7fc02c00c6a0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 shutdown_connections 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0200a2970 0x7fc0200a4e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034106710 0x7fc03419c9b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0341010c0 0x7fc034198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 --2- 172.21.15.49:0/1957319629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034100700 0x7fc0341980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 >> 172.21.15.49:0/1957319629 conn(0x7fc034077840 msgr2=0x7fc03406a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 shutdown_connections 2024-03-16T08:21:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:56.876+0000 7fc03c597640 1 -- 172.21.15.49:0/1957319629 wait complete. 2024-03-16T08:21:57.888 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:21:57.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:21:57.888 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 -- 172.21.15.49:0/1541900496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684107680 msgr2=0x7f8684107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 --2- 172.21.15.49:0/1541900496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684107680 0x7f8684107a60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8674009930 tx=0x7f867402bc30 comp rx=0 tx=0).stop 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 -- 172.21.15.49:0/1541900496 shutdown_connections 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 --2- 172.21.15.49:0/1541900496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f868410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 --2- 172.21.15.49:0/1541900496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684101670 0x7f8684101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 --2- 172.21.15.49:0/1541900496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684107680 0x7f8684107a60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 -- 172.21.15.49:0/1541900496 >> 172.21.15.49:0/1541900496 conn(0x7f86840fd180 msgr2=0x7f86840ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 -- 172.21.15.49:0/1541900496 shutdown_connections 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.012+0000 7f8688800640 1 -- 172.21.15.49:0/1541900496 wait complete. 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 Processor -- start 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 -- start start 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 0x7f8684197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684107680 0x7f868419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8684111a90 con 0x7f8684101670 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8684111910 con 0x7f8684107680 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8688800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8684111c10 con 0x7f8684102030 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.013+0000 7f8682d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 0x7f8684197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:48832/0 (socket says 172.21.15.49:48832) 2024-03-16T08:21:58.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684107680 0x7f868419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 -- 172.21.15.49:0/3305255677 learned_addr learned my addr 172.21.15.49:0/3305255677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f8682d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 0x7f8684197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60734/0 (socket says 172.21.15.49:60734) 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 -- 172.21.15.49:0/3305255677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684107680 msgr2=0x7f868419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684107680 0x7f868419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 -- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 msgr2=0x7f8684197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 0x7f8684197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 -- 172.21.15.49:0/3305255677 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868419cf60 con 0x7f8684102030 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f868259c640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9f0 tx=0x7f867800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f866bfff640 1 -- 172.21.15.49:0/3305255677 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867800cd80 con 0x7f8684102030 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f866bfff640 1 -- 172.21.15.49:0/3305255677 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8678010910 con 0x7f8684102030 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868419d250 con 0x7f8684102030 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f866bfff640 1 -- 172.21.15.49:0/3305255677 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678018990 con 0x7f8684102030 2024-03-16T08:21:58.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.014+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86841a4ae0 con 0x7f8684102030 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.016+0000 7f866bfff640 1 -- 172.21.15.49:0/3305255677 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8678010ac0 con 0x7f8684102030 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 Processor -- start 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 -- start start 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f866bfff640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 0x7f866c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 0x561814b72070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868359e640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 0x7f866c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561814ad50d0 con 0x561814b7af60 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561814a7dc70 con 0x561814b728d0 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.017+0000 7f868bbea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561814ad1170 con 0x561814b71bf0 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f866bfff640 1 -- 172.21.15.49:0/3305255677 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f86780cf150 con 0x7f8684102030 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f868359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41704/0 (socket says 172.21.15.49:41704) 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 -- 172.21.15.49:0/1057389346 learned_addr learned my addr 172.21.15.49:0/1057389346 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f868259c640 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 0x561814b72070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 -- 172.21.15.49:0/1057389346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 msgr2=0x561814b72070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 0x561814b72070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 -- 172.21.15.49:0/1057389346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 msgr2=0x561814b7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f868359e640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 0x7f866c0a4e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c013ae0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 -- 172.21.15.49:0/1057389346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618148b17e0 con 0x561814b728d0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f868259c640 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 0x561814b72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f8682d9d640 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f867402cfd0 tx=0x7f86740095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f9466fb11d2cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f86697fa640 1 -- 172.21.15.49:0/1057389346 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674042070 con 0x561814b728d0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.018+0000 7f86697fa640 1 -- 172.21.15.49:0/1057389346 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8674004030 con 0x561814b728d0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f86697fa640 1 -- 172.21.15.49:0/1057389346 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674004bf0 con 0x561814b728d0 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- 172.21.15.49:0/1057389346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 msgr2=0x561814b71620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f867402cfd0 tx=0x7f86740095f0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- 172.21.15.49:0/1057389346 shutdown_connections 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b71bf0 0x561814b72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 --2- 172.21.15.49:0/1057389346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b728d0 0x561814b71620 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- 172.21.15.49:0/1057389346 >> 172.21.15.49:0/1057389346 conn(0x561814b38c50 msgr2=0x561814b6c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- 172.21.15.49:0/1057389346 shutdown_connections 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- 172.21.15.49:0/1057389346 wait complete. 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.019+0000 7f868bbea740 1 -- start start 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 Processor -- start 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561814b7dd70 con 0x561814b7af60 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561814a910d0 con 0x561814b71bf0 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561814b7def0 con 0x561814b728d0 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868bbea740 10 client.0 _ll_register_callbacks cb 0x561814a5a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60748/0 (socket says 172.21.15.49:60748) 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f8682d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 -- 172.21.15.49:0/2010675063 learned_addr learned my addr 172.21.15.49:0/2010675063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868259c640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 -- 172.21.15.49:0/2010675063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 msgr2=0x561814b784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 -- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 msgr2=0x561814b77f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b77f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868359e640 1 -- 172.21.15.49:0/2010675063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561814911f70 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f868259c640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.020+0000 7f8682d9d640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868359e640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f867c003c20 tx=0x7f867c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eeb42a1b5eae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c004460 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f867c004600 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8674004030 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f867c02e020 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 10 client.16785 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618147a9170 con 0x561814b7af60 2024-03-16T08:21:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8674004030 con 0x561814b7af60 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f867c033430 con 0x561814b7af60 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 10 client.16785 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 20 client.16785 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814b8b000 0x561814b8d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 4 client.16785 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.021+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561814ae2610 con 0x561814b8b000 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.022+0000 7f8682d9d640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814b8b000 0x561814b8d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.022+0000 7f8682d9d640 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814b8b000 0x561814b8d430 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b7bc67d6877015 server_cookie=405a31744161f023 in_seq=0 out_seq=0 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.022+0000 7f8666ff5640 10 client.16785 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f8666ff5640 1 -- 172.21.15.49:0/2010675063 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561814ae2610 con 0x561814b8b000 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f8666ff5640 10 client.16785 handle_command_reply: tid=0 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 client.16785 shutdown 2024-03-16T08:21:58.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 msgr2=0x561814b789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f867c003c20 tx=0x7f867c004ab0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 shutdown_connections 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814b8b000 0x561814b8d430 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b789e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f8666ff5640 0 client.16785 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 --2- 172.21.15.49:0/2010675063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 >> 172.21.15.49:0/2010675063 conn(0x561814b38c50 msgr2=0x561814b6e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 shutdown_connections 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 1 -- 172.21.15.49:0/2010675063 wait complete. 2024-03-16T08:21:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.023+0000 7f868bbea740 20 client.16785 trim_cache size 0 max 16384 2024-03-16T08:21:58.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 Processor -- start 2024-03-16T08:21:58.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 -- start start 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b7af60 0x561814b92d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86841afaa0 con 0x561814b728d0 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86841ad370 con 0x561814b71bf0 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868bbea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86841ab1c0 con 0x561814b7af60 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60764/0 (socket says 172.21.15.49:60764) 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 -- 172.21.15.49:0/3260731607 learned_addr learned my addr 172.21.15.49:0/3260731607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f8682d9d640 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868359e640 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b7af60 0x561814b92d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 -- 172.21.15.49:0/3260731607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b7af60 msgr2=0x561814b92d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b7af60 0x561814b92d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 -- 172.21.15.49:0/3260731607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 msgr2=0x561814b7a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b7a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.041+0000 7f868259c640 1 -- 172.21.15.49:0/3260731607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561814b70c20 con 0x561814b728d0 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868259c640 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f867809b5b0 tx=0x7f86780ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2450f430ec5c1011 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f8668ff9640 1 -- 172.21.15.49:0/3260731607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86780b3020 con 0x561814b728d0 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f8668ff9640 1 -- 172.21.15.49:0/3260731607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8678097f20 con 0x561814b728d0 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f8668ff9640 1 -- 172.21.15.49:0/3260731607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678018b30 con 0x561814b728d0 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 -- 172.21.15.49:0/3260731607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 msgr2=0x561814b86b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f867809b5b0 tx=0x7f86780ce040 comp rx=0 tx=0).stop 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 -- 172.21.15.49:0/3260731607 shutdown_connections 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b7af60 0x561814b92d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b728d0 0x561814b86b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 --2- 172.21.15.49:0/3260731607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 -- 172.21.15.49:0/3260731607 >> 172.21.15.49:0/3260731607 conn(0x561814b38c50 msgr2=0x561814b77f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.042+0000 7f868bbea740 1 -- 172.21.15.49:0/3260731607 shutdown_connections 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 -- 172.21.15.49:0/3260731607 wait complete. 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 -- start start 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 Processor -- start 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86841ab830 con 0x561814b7af60 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86841acca0 con 0x561814b71bf0 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86841abf00 con 0x561814b728d0 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868bbea740 10 client.0 _ll_register_callbacks cb 0x7f867c012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60772/0 (socket says 172.21.15.49:60772) 2024-03-16T08:21:58.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f868359e640 1 -- 172.21.15.49:0/498103459 learned_addr learned my addr 172.21.15.49:0/498103459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.043+0000 7f8682d9d640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 -- 172.21.15.49:0/498103459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 msgr2=0x561814b75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b75a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 -- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 msgr2=0x561814b754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 -- 172.21.15.49:0/498103459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561814b83910 con 0x561814b7af60 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868259c640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8682d9d640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868359e640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f867c0338d0 tx=0x7f867c0134e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8360a4d69be2d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c003a30 con 0x561814b7af60 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f867c033cd0 con 0x561814b7af60 2024-03-16T08:21:58.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8678097f20 con 0x561814b7af60 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f867c004260 con 0x561814b7af60 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868bbea740 10 client.16791 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561814bf1920 con 0x561814b7af60 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8678097f20 con 0x561814b7af60 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.044+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f867c047860 con 0x561814b7af60 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 10 client.16791 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 20 client.16791 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814bf2050 0x561814bf4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 4 client.16791 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561814adc960 con 0x561814bf2050 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f8682d9d640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814bf2050 0x561814bf4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f8682d9d640 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814bf2050 0x561814bf4470 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61181f3369244e6 server_cookie=31a931f66f7a4afe in_seq=0 out_seq=0 2024-03-16T08:21:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.045+0000 7f8666ff5640 10 client.16791 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f8666ff5640 1 -- 172.21.15.49:0/498103459 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561814adc960 con 0x561814bf2050 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f8666ff5640 10 client.16791 handle_command_reply: tid=0 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 client.16791 shutdown 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 msgr2=0x561814b7bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f867c0338d0 tx=0x7f867c0134e0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 shutdown_connections 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561814bf2050 0x561814bf4470 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561814b7af60 0x561814b7bc90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561814b728d0 0x561814b75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 --2- 172.21.15.49:0/498103459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561814b71bf0 0x561814b754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.046+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 >> 172.21.15.49:0/498103459 conn(0x561814b38c50 msgr2=0x561814b787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8666ff5640 0 client.16791 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 shutdown_connections 2024-03-16T08:21:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f868bbea740 1 -- 172.21.15.49:0/498103459 wait complete. 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f868bbea740 20 client.16791 trim_cache size 0 max 16384 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "99c60bff-2e0d-4f36-b3be-335624704915", 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:21:58.054 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 msgr2=0x7f866c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 0x7f866c0a4e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c013ae0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 msgr2=0x7f86841984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9f0 tx=0x7f867800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 shutdown_connections 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f866c0a2970 0x7f866c0a4e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8684107680 0x7f868419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8684102030 0x7f86841984b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 --2- 172.21.15.49:0/3305255677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8684101670 0x7f8684197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 >> 172.21.15.49:0/3305255677 conn(0x7f86840fd180 msgr2=0x7f86840fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 shutdown_connections 2024-03-16T08:21:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.047+0000 7f8688800640 1 -- 172.21.15.49:0/3305255677 wait complete. 2024-03-16T08:21:58.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:21:58.060 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:21:58.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 -- 172.21.15.49:0/2472283866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc103390 msgr2=0x7f0edc10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/2472283866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc103390 0x7f0edc10b870 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009900 tx=0x7f0ed002bcd0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 -- 172.21.15.49:0/2472283866 shutdown_connections 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/2472283866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc10be40 0x7f0edc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/2472283866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc103390 0x7f0edc10b870 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/2472283866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc102a70 0x7f0edc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 -- 172.21.15.49:0/2472283866 >> 172.21.15.49:0/2472283866 conn(0x7f0edc077840 msgr2=0x7f0edc077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 -- 172.21.15.49:0/2472283866 shutdown_connections 2024-03-16T08:21:58.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.169+0000 7f0ee29e6640 1 -- 172.21.15.49:0/2472283866 wait complete. 2024-03-16T08:21:58.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 Processor -- start 2024-03-16T08:21:58.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 -- start start 2024-03-16T08:21:58.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc102a70 0x7f0edc1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc103390 0x7f0edc100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc111be0 con 0x7f0edc10be40 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc111a60 con 0x7f0edc103390 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee29e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc111d60 con 0x7f0edc102a70 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60782/0 (socket says 172.21.15.49:60782) 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 -- 172.21.15.49:0/3897368883 learned_addr learned my addr 172.21.15.49:0/3897368883 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc103390 0x7f0edc100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0edbfff640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc102a70 0x7f0edc1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 -- 172.21.15.49:0/3897368883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc102a70 msgr2=0x7f0edc1006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc102a70 0x7f0edc1006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 -- 172.21.15.49:0/3897368883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc103390 msgr2=0x7f0edc100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.170+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc103390 0x7f0edc100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ee0f83640 1 -- 172.21.15.49:0/3897368883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc1a65a0 con 0x7f0edc10be40 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400eb10 tx=0x7f0ed400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ed97fa640 1 -- 172.21.15.49:0/3897368883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004b20 con 0x7f0edc10be40 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc1a6890 con 0x7f0edc10be40 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ed97fa640 1 -- 172.21.15.49:0/3897368883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed4004cc0 con 0x7f0edc10be40 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc1a6e00 con 0x7f0edc10be40 2024-03-16T08:21:58.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.171+0000 7f0ed97fa640 1 -- 172.21.15.49:0/3897368883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4010700 con 0x7f0edc10be40 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.173+0000 7f0ed97fa640 1 -- 172.21.15.49:0/3897368883 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0ed4005020 con 0x7f0edc10be40 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 Processor -- start 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 -- start start 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ed97fa640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 0x7f0ec40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0ba70 0x55b1fca14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1fc96f500 con 0x55b1fca14650 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b1fc917be0 con 0x55b1fca0b150 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1fc96b5a0 con 0x55b1fca0ba70 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 0x7f0ec40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41754/0 (socket says 172.21.15.49:41754) 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0edbfff640 1 -- 172.21.15.49:0/688336761 learned_addr learned my addr 172.21.15.49:0/688336761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0ba70 0x55b1fca14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.174+0000 7f0ed97fa640 1 -- 172.21.15.49:0/3897368883 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0ed40cf480 con 0x7f0edc10be40 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 -- 172.21.15.49:0/688336761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0ba70 msgr2=0x55b1fca14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0ba70 0x55b1fca14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 -- 172.21.15.49:0/688336761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 msgr2=0x55b1fca16a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca16a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 -- 172.21.15.49:0/688336761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1fc92b4b0 con 0x55b1fca0b150 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 0x7f0ec40a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed002efd0 tx=0x7f0ed00095d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0edbfff640 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009cf0 tx=0x7f0ecc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6daaa6960694608 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0ec27fc640 1 -- 172.21.15.49:0/688336761 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc004220 con 0x55b1fca0b150 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0ec27fc640 1 -- 172.21.15.49:0/688336761 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc0043c0 con 0x55b1fca0b150 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.175+0000 7f0ec27fc640 1 -- 172.21.15.49:0/688336761 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc018910 con 0x55b1fca0b150 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- 172.21.15.49:0/688336761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 msgr2=0x55b1fca0b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009cf0 tx=0x7f0ecc011810 comp rx=0 tx=0).stop 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- 172.21.15.49:0/688336761 shutdown_connections 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0ba70 0x55b1fca14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/688336761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0b150 0x55b1fca0b530 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- 172.21.15.49:0/688336761 >> 172.21.15.49:0/688336761 conn(0x55b1fca05880 msgr2=0x55b1fca05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- 172.21.15.49:0/688336761 shutdown_connections 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- 172.21.15.49:0/688336761 wait complete. 2024-03-16T08:21:58.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.176+0000 7f0ee55d3740 1 -- start start 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 Processor -- start 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 0x55b1fca116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca12140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1fca17440 con 0x55b1fca14650 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b1fc92b160 con 0x55b1fca0ba70 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1fca175c0 con 0x55b1fca0b150 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee55d3740 10 client.0 _ll_register_callbacks cb 0x55b1fc90f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0edb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0ee0f83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca12140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 0x55b1fca116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0edb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:41760/0 (socket says 172.21.15.49:41760) 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.177+0000 7f0edb7fe640 1 -- 172.21.15.49:0/3259960413 learned_addr learned my addr 172.21.15.49:0/3259960413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 -- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 msgr2=0x55b1fca11c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 -- 172.21.15.49:0/3259960413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 msgr2=0x55b1fca12140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca12140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 -- 172.21.15.49:0/3259960413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1fc74d7d0 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca12140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0edbfff640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 0x55b1fca116c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009cf0 tx=0x7f0ecc005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ec2ca3592001c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ebb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc026070 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0ecc003800 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ecc0043c0 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0ecc032050 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ee55d3740 10 client.26686 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.178+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1fc6833f0 con 0x55b1fca0b150 2024-03-16T08:21:58.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc0043c0 con 0x55b1fca0b150 2024-03-16T08:21:58.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0ecc019430 con 0x55b1fca0b150 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 10 client.26686 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 20 client.26686 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca244f0 0x55b1fca268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 4 client.26686 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1fc97ca40 con 0x55b1fca244f0 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.179+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca244f0 0x55b1fca268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.180+0000 7f0ebb7fe640 10 client.26686 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.180+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca244f0 0x55b1fca268e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1de9078ae472627 server_cookie=73704d36d5fb358e in_seq=0 out_seq=0 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/3259960413 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b1fc97ca40 con 0x55b1fca244f0 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ebb7fe640 10 client.26686 handle_command_reply: tid=0 2024-03-16T08:21:58.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 client.26686 shutdown 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 msgr2=0x55b1fca116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 0x55b1fca116c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009cf0 tx=0x7f0ecc005760 comp rx=0 tx=0).stop 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 shutdown_connections 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca244f0 0x55b1fca268e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca14650 0x55b1fca12140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ebb7fe640 0 client.26686 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca11c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/3259960413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca0b150 0x55b1fca116c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 >> 172.21.15.49:0/3259960413 conn(0x55b1fca05880 msgr2=0x55b1fca075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 shutdown_connections 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 1 -- 172.21.15.49:0/3259960413 wait complete. 2024-03-16T08:21:58.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.181+0000 7f0ee55d3740 20 client.26686 trim_cache size 0 max 16384 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.199+0000 7f0ee55d3740 1 Processor -- start 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.199+0000 7f0ee55d3740 1 -- start start 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca2c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1b1ec0 con 0x55b1fca0b150 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1af790 con 0x55b1fca0ba70 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1ad5e0 con 0x55b1fca14650 2024-03-16T08:21:58.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60816/0 (socket says 172.21.15.49:60816) 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 -- 172.21.15.49:0/1240544529 learned_addr learned my addr 172.21.15.49:0/1240544529 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca2c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 -- 172.21.15.49:0/1240544529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 msgr2=0x55b1fca2c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca2c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 -- 172.21.15.49:0/1240544529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 msgr2=0x55b1fca201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 -- 172.21.15.49:0/1240544529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1fca26e80 con 0x55b1fca0b150 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0edbfff640 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009560 tx=0x7f0ecc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834b5f5d8598388 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ec1ffb640 1 -- 172.21.15.49:0/1240544529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc026040 con 0x55b1fca0b150 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ec1ffb640 1 -- 172.21.15.49:0/1240544529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc015070 con 0x55b1fca0b150 2024-03-16T08:21:58.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.200+0000 7f0ec1ffb640 1 -- 172.21.15.49:0/1240544529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc019430 con 0x55b1fca0b150 2024-03-16T08:21:58.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- 172.21.15.49:0/1240544529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 msgr2=0x55b1fca13df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009560 tx=0x7f0ecc009640 comp rx=0 tx=0).stop 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- 172.21.15.49:0/1240544529 shutdown_connections 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/1240544529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca13df0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- 172.21.15.49:0/1240544529 >> 172.21.15.49:0/1240544529 conn(0x55b1fca05880 msgr2=0x55b1fca05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- 172.21.15.49:0/1240544529 shutdown_connections 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- 172.21.15.49:0/1240544529 wait complete. 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.201+0000 7f0ee55d3740 1 -- start start 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 Processor -- start 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca17dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca18310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1adc50 con 0x55b1fca0b150 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1af0c0 con 0x55b1fca0ba70 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc1ae320 con 0x55b1fca14650 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee55d3740 10 client.0 _ll_register_callbacks cb 0x7f0ecc009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca17dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60826/0 (socket says 172.21.15.49:60826) 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edbfff640 1 -- 172.21.15.49:0/45355068 learned_addr learned my addr 172.21.15.49:0/45355068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0ee0f83640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca18310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 -- 172.21.15.49:0/45355068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 msgr2=0x55b1fca18310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca18310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 -- 172.21.15.49:0/45355068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 msgr2=0x55b1fca1c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edb7fe640 1 -- 172.21.15.49:0/45355068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1fca0b8a0 con 0x55b1fca0ba70 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.202+0000 7f0edbfff640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:21:58.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0edb7fe640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca17dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006010 tx=0x7f0ed002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ecf9bf784d51a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0053350 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f0ed00534f0 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ecc015070 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0ed00536f0 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 10 client.26690 fetch_fsmap learned FSMap version 24 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1fc6430f0 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc015070 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0ed004d040 con 0x55b1fca0ba70 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 10 client.26690 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 20 client.26690 populate_metadata read hostname 'smithi049' 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca8a950 0x55b1fca8cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:21:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 4 client.26690 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1fc976d90 con 0x55b1fca8a950 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.203+0000 7f0edbfff640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca8a950 0x55b1fca8cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ebb7fe640 10 client.26690 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0edbfff640 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca8a950 0x55b1fca8cd70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b19c7b906b2cad server_cookie=69687816eda83004 in_seq=0 out_seq=0 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ebb7fe640 1 -- 172.21.15.49:0/45355068 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1fc976d90 con 0x55b1fca8a950 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ebb7fe640 10 client.26690 handle_command_reply: tid=0 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 client.26690 shutdown 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 msgr2=0x55b1fca17dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca17dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006010 tx=0x7f0ed002bcd0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 shutdown_connections 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b1fca8a950 0x55b1fca8cd70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1fca14650 0x55b1fca18310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1fca0ba70 0x55b1fca17dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 --2- 172.21.15.49:0/45355068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1fca0b150 0x55b1fca1c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ebb7fe640 0 client.26690 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 >> 172.21.15.49:0/45355068 conn(0x55b1fca05880 msgr2=0x55b1fca04070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 shutdown_connections 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 1 -- 172.21.15.49:0/45355068 wait complete. 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.204+0000 7f0ee55d3740 20 client.26690 trim_cache size 0 max 16384 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stdout: "99c60bff-2e0d-4f36-b3be-335624704915": { 2024-03-16T08:21:58.212 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "99c60bff-2e0d-4f36-b3be-335624704915", 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:21:58.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 msgr2=0x7f0ec40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 0x7f0ec40a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed002efd0 tx=0x7f0ed00095d0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 msgr2=0x7f0edc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400eb10 tx=0x7f0ed400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 shutdown_connections 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0ec40a2970 0x7f0ec40a4e30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0edc10be40 0x7f0edc101140 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0edc103390 0x7f0edc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 --2- 172.21.15.49:0/3897368883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0edc102a70 0x7f0edc1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 >> 172.21.15.49:0/3897368883 conn(0x7f0edc077840 msgr2=0x7f0edc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 shutdown_connections 2024-03-16T08:21:58.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:21:58.205+0000 7f0ee29e6640 1 -- 172.21.15.49:0/3897368883 wait complete. 2024-03-16T08:21:58.217 INFO:tasks.cephfs.filesystem:scrub status for tag:99c60bff-2e0d-4f36-b3be-335624704915 - {'path': '/', 'tag': '99c60bff-2e0d-4f36-b3be-335624704915', 'options': 'recursive,force'} 2024-03-16T08:21:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:58 smithi049 ceph-mon[31942]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:58.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:58 smithi154 ceph-mon[36471]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:58.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:58 smithi129 ceph-mon[36561]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:21:59.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:59 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:21:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:59 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:59 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:21:59.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:21:59 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:59 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:21:59.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:59 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:59.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:59 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:21:59.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:21:59 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:59.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:59 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:21:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:59 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:21:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:59 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:21:59.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:21:59 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:00.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:00 smithi154 ceph-mon[36471]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:00.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:00 smithi049 ceph-mon[31942]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:00 smithi129 ceph-mon[36561]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:02.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:02 smithi154 ceph-mon[36471]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:02.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:02 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:22:02.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:02 smithi049 ceph-mon[31942]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:02.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:02 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:22:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:02 smithi129 ceph-mon[36561]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:02 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:22:04.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:04 smithi154 ceph-mon[36471]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:04 smithi049 ceph-mon[31942]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:04.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:04 smithi129 ceph-mon[36561]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:06.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:06 smithi154 ceph-mon[36471]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:06.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:06 smithi049 ceph-mon[31942]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:06 smithi129 ceph-mon[36561]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:08.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:08 smithi154 ceph-mon[36471]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:22:08.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:08 smithi049 ceph-mon[31942]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:22:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:08 smithi129 ceph-mon[36561]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:22:10.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:10 smithi154 ceph-mon[36471]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:10.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:10 smithi049 ceph-mon[31942]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:10.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:10 smithi129 ceph-mon[36561]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:22:12.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:12 smithi154 ceph-mon[36471]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:12.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:12 smithi049 ceph-mon[31942]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:12 smithi129 ceph-mon[36561]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:14.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:14 smithi154 ceph-mon[36471]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:14.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:14 smithi049 ceph-mon[31942]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:14.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:14 smithi129 ceph-mon[36561]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-16T08:22:16.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:16 smithi154 ceph-mon[36471]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:22:16.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:16 smithi049 ceph-mon[31942]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:22:16.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:16 smithi129 ceph-mon[36561]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-16T08:22:18.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:18 smithi154 ceph-mon[36471]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:22:18.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:18 smithi049 ceph-mon[31942]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:22:18.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:18 smithi129 ceph-mon[36561]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:22:20.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:20 smithi154 ceph-mon[36471]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:22:20.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:20 smithi049 ceph-mon[31942]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:22:20.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:20 smithi129 ceph-mon[36561]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:22:22.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:22 smithi154 ceph-mon[36471]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:22.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:22 smithi049 ceph-mon[31942]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:22.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:22 smithi129 ceph-mon[36561]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:23.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:22:23.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:22:23.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:22:23.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:23 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:22:23.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:22:23.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:22:23.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:22:23.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:23 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:22:23.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:22:23.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:22:23.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:22:23.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:23 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:22:24.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:24 smithi049 ceph-mon[31942]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:24.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:24 smithi129 ceph-mon[36561]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:24.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:24 smithi154 ceph-mon[36471]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:26.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:26 smithi049 ceph-mon[31942]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:26.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:26 smithi129 ceph-mon[36561]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:26.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:26 smithi154 ceph-mon[36471]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:28.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:22:28.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 -- 172.21.15.49:0/1276532043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 msgr2=0x7f7f40106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/1276532043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f40106af0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009930 tx=0x7f7f3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 -- 172.21.15.49:0/1276532043 shutdown_connections 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/1276532043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f4010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/1276532043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f40100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/1276532043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f40106af0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 -- 172.21.15.49:0/1276532043 >> 172.21.15.49:0/1276532043 conn(0x7f7f40077840 msgr2=0x7f7f40077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 -- 172.21.15.49:0/1276532043 shutdown_connections 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.341+0000 7f7f46bdf640 1 -- 172.21.15.49:0/1276532043 wait complete. 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 Processor -- start 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 -- start start 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f40198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40110b20 con 0x7f7f40106710 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401109a0 con 0x7f7f401010c0 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f46bdf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40110ca0 con 0x7f7f40100700 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f4517c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f4517c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35600/0 (socket says 172.21.15.49:35600) 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f40198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.342+0000 7f7f4517c640 1 -- 172.21.15.49:0/2431141254 learned_addr learned my addr 172.21.15.49:0/2431141254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 -- 172.21.15.49:0/2431141254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 msgr2=0x7f7f401981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f401981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 -- 172.21.15.49:0/2431141254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 msgr2=0x7f7f40198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f40198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 -- 172.21.15.49:0/2431141254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4019d1e0 con 0x7f7f40106710 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f40198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f4517c640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800eb10 tx=0x7f7f3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f3dffb640 1 -- 172.21.15.49:0/2431141254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004b20 con 0x7f7f40106710 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f3dffb640 1 -- 172.21.15.49:0/2431141254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f38004cc0 con 0x7f7f40106710 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4019d4d0 con 0x7f7f40106710 2024-03-16T08:22:28.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f3dffb640 1 -- 172.21.15.49:0/2431141254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38010760 con 0x7f7f40106710 2024-03-16T08:22:28.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.343+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f401a4d60 con 0x7f7f40106710 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.345+0000 7f7f3dffb640 1 -- 172.21.15.49:0/2431141254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7f38005020 con 0x7f7f40106710 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 Processor -- start 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 -- start start 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f3dffb640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 0x7f7f2c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fec262b500 con 0x55fec26c7a70 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fec25d3be0 con 0x55fec26d0650 2024-03-16T08:22:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f497c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fec26275a0 con 0x55fec26c7150 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f3dffb640 1 -- 172.21.15.49:0/2431141254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f380cf320 con 0x7f7f40106710 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 0x7f7f2c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36278/0 (socket says 172.21.15.49:36278) 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4497b640 1 -- 172.21.15.49:0/1160120937 learned_addr learned my addr 172.21.15.49:0/1160120937 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 -- 172.21.15.49:0/1160120937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 msgr2=0x55fec26c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 -- 172.21.15.49:0/1160120937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 msgr2=0x55fec26d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4517c640 1 -- 172.21.15.49:0/1160120937 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec25e74b0 con 0x55fec26d0650 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.346+0000 7f7f4497b640 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.347+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 0x7f7f2c0a4e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55fec26c6c20 tx=0x7f7f28004730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.347+0000 7f7f4517c640 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26d2a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f380ccac0 tx=0x7f7f3809b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e9917764f5f543 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.347+0000 7f7f36ffd640 1 -- 172.21.15.49:0/1160120937 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3809c0d0 con 0x55fec26d0650 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.347+0000 7f7f36ffd640 1 -- 172.21.15.49:0/1160120937 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3809b9d0 con 0x55fec26d0650 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.347+0000 7f7f36ffd640 1 -- 172.21.15.49:0/1160120937 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38021440 con 0x55fec26d0650 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- 172.21.15.49:0/1160120937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 msgr2=0x55fec26d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26d2a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f380ccac0 tx=0x7f7f3809b510 comp rx=0 tx=0).stop 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- 172.21.15.49:0/1160120937 shutdown_connections 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26d2a40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 --2- 172.21.15.49:0/1160120937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7150 0x55fec26c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- 172.21.15.49:0/1160120937 >> 172.21.15.49:0/1160120937 conn(0x55fec26c1880 msgr2=0x55fec26c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- 172.21.15.49:0/1160120937 shutdown_connections 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- 172.21.15.49:0/1160120937 wait complete. 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 -- start start 2024-03-16T08:22:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 Processor -- start 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.348+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 0x55fec26cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fec26d33e0 con 0x55fec26c7150 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fec25e7160 con 0x55fec26c7a70 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fec26d3560 con 0x55fec26d0650 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 10 client.0 _ll_register_callbacks cb 0x55fec25cb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35624/0 (socket says 172.21.15.49:35624) 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 -- 172.21.15.49:0/3163036811 learned_addr learned my addr 172.21.15.49:0/3163036811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 0x55fec26cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4517c640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 -- 172.21.15.49:0/3163036811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 msgr2=0x55fec26ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 -- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 msgr2=0x55fec26cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 0x55fec26cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 -- 172.21.15.49:0/3163036811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec24097d0 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 0x55fec26cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f4497b640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009230 tx=0x7f7f3002cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e48e7cfcfbd35a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004040 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f300041e0 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.349+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3809b9d0 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f300043e0 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 10 client.16821 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fec233f3f0 con 0x55fec26c7150 2024-03-16T08:22:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3809b9d0 con 0x55fec26c7150 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7f300038c0 con 0x55fec26c7150 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 10 client.16821 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 20 client.16821 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec26e0690 0x55fec26e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 4 client.16821 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fec2638a40 con 0x55fec26e0690 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.350+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec26e0690 0x55fec26e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.351+0000 7f7f1ffff640 10 client.16821 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.351+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec26e0690 0x55fec26e2b10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903789349b45e626 server_cookie=c41f0b5d93cd7b30 in_seq=0 out_seq=0 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f1ffff640 1 -- 172.21.15.49:0/3163036811 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fec2638a40 con 0x55fec26e0690 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f1ffff640 10 client.16821 handle_command_reply: tid=0 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 client.16821 shutdown 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 msgr2=0x55fec26cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009230 tx=0x7f7f3002cab0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 shutdown_connections 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec26e0690 0x55fec26e2b10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7a70 0x55fec26cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f1ffff640 0 client.16821 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 --2- 172.21.15.49:0/3163036811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cd6f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 >> 172.21.15.49:0/3163036811 conn(0x55fec26c1880 msgr2=0x55fec26c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 shutdown_connections 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 1 -- 172.21.15.49:0/3163036811 wait complete. 2024-03-16T08:22:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.352+0000 7f7f497c6740 20 client.16821 trim_cache size 0 max 16384 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 Processor -- start 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 -- start start 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7a70 0x55fec26cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401afce0 con 0x55fec26c7150 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401ad5b0 con 0x55fec26d0650 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f497c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401ab400 con 0x55fec26c7a70 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35636/0 (socket says 172.21.15.49:35636) 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 -- 172.21.15.49:0/2258623400 learned_addr learned my addr 172.21.15.49:0/2258623400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4517c640 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7a70 0x55fec26cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 -- 172.21.15.49:0/2258623400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7a70 msgr2=0x55fec26cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7a70 0x55fec26cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 -- 172.21.15.49:0/2258623400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 msgr2=0x55fec26c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 -- 172.21.15.49:0/2258623400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec22ff0f0 con 0x55fec26c7150 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.370+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7f30003d60 tx=0x7f7f3002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c392c0bea1108baa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f367fc640 1 -- 172.21.15.49:0/2258623400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30054070 con 0x55fec26c7150 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f367fc640 1 -- 172.21.15.49:0/2258623400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f30034440 con 0x55fec26c7150 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f367fc640 1 -- 172.21.15.49:0/2258623400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004240 con 0x55fec26c7150 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- 172.21.15.49:0/2258623400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 msgr2=0x55fec26cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7f30003d60 tx=0x7f7f3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- 172.21.15.49:0/2258623400 shutdown_connections 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26d0650 0x55fec26c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26c7a70 0x55fec26cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2258623400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7150 0x55fec26cfe20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- 172.21.15.49:0/2258623400 >> 172.21.15.49:0/2258623400 conn(0x55fec26c1880 msgr2=0x55fec26c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- 172.21.15.49:0/2258623400 shutdown_connections 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- 172.21.15.49:0/2258623400 wait complete. 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.371+0000 7f7f497c6740 1 -- start start 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 Processor -- start 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401aba70 con 0x55fec26c7a70 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401acee0 con 0x55fec26c7150 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401ac140 con 0x55fec26d0650 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f497c6740 10 client.0 _ll_register_callbacks cb 0x7f7f3002c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f4497b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42416/0 (socket says 172.21.15.49:42416) 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35646/0 (socket says 172.21.15.49:35646) 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f4497b640 1 -- 172.21.15.49:0/2782638239 learned_addr learned my addr 172.21.15.49:0/2782638239 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f4517c640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 -- 172.21.15.49:0/2782638239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 msgr2=0x55fec26d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 -- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 msgr2=0x55fec26d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f3ffff640 1 -- 172.21.15.49:0/2782638239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec25f29d0 con 0x55fec26c7a70 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.372+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f3ffff640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55fec26e3350 tx=0x7f7f280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4733d5801b3d86a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28019040 con 0x55fec26c7a70 2024-03-16T08:22:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fec2632d90 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7f2802d680 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f2803d050 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 10 client.16827 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fec2746ea0 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fec2632d90 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7f2802a050 con 0x55fec26c7a70 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 10 client.16827 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 20 client.16827 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec2747570 0x55fec2749990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 4 client.16827 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.373+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fec25b0540 con 0x55fec2747570 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec2747570 0x55fec2749990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f1ffff640 10 client.16827 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f4497b640 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec2747570 0x55fec2749990 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15082ecdc45148c6 server_cookie=c1910a3bac2cbebd in_seq=0 out_seq=0 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f1ffff640 1 -- 172.21.15.49:0/2782638239 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fec25b0540 con 0x55fec2747570 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f1ffff640 10 client.16827 handle_command_reply: tid=0 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 client.16827 shutdown 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 msgr2=0x55fec26d48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55fec26e3350 tx=0x7f7f280046c0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 shutdown_connections 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fec2747570 0x55fec2749990 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fec26d0650 0x55fec26d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fec26c7a70 0x55fec26d48d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f1ffff640 0 client.16827 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 --2- 172.21.15.49:0/2782638239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fec26c7150 0x55fec26d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 >> 172.21.15.49:0/2782638239 conn(0x55fec26c1880 msgr2=0x55fec26c0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 shutdown_connections 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.374+0000 7f7f497c6740 1 -- 172.21.15.49:0/2782638239 wait complete. 2024-03-16T08:22:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f497c6740 20 client.16827 trim_cache size 0 max 16384 2024-03-16T08:22:28.383 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:22:28.383 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:22:28.383 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:22:28.383 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 msgr2=0x7f7f2c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 0x7f7f2c0a4e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55fec26c6c20 tx=0x7f7f28004730 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 msgr2=0x7f7f4019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800eb10 tx=0x7f7f3800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 shutdown_connections 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7f2c0a2970 0x7f7f2c0a4e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f40106710 0x7f7f4019cab0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f401010c0 0x7f7f40198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 --2- 172.21.15.49:0/2431141254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f40100700 0x7f7f401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 >> 172.21.15.49:0/2431141254 conn(0x7f7f40077840 msgr2=0x7f7f4006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 shutdown_connections 2024-03-16T08:22:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.375+0000 7f7f46bdf640 1 -- 172.21.15.49:0/2431141254 wait complete. 2024-03-16T08:22:28.387 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:22:28.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:22:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 -- 172.21.15.49:0/2051763728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b0101060 msgr2=0x7f14b010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 --2- 172.21.15.49:0/2051763728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b0101060 0x7f14b010d570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009be0 tx=0x7f14a802bef0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 -- 172.21.15.49:0/2051763728 shutdown_connections 2024-03-16T08:22:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 --2- 172.21.15.49:0/2051763728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b0101060 0x7f14b010d570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 --2- 172.21.15.49:0/2051763728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01006a0 0x7f14b0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 --2- 172.21.15.49:0/2051763728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b01066b0 0x7f14b0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 -- 172.21.15.49:0/2051763728 >> 172.21.15.49:0/2051763728 conn(0x7f14b0077780 msgr2=0x7f14b0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 -- 172.21.15.49:0/2051763728 shutdown_connections 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 -- 172.21.15.49:0/2051763728 wait complete. 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.495+0000 7f14b8787640 1 Processor -- start 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 -- start start 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b01006a0 0x7f14b0102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b0101060 0x7f14b01027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0110ac0 con 0x7f14b01066b0 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0110940 con 0x7f14b01006a0 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b8787640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0110c40 con 0x7f14b0101060 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35674/0 (socket says 172.21.15.49:35674) 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 -- 172.21.15.49:0/987161157 learned_addr learned my addr 172.21.15.49:0/987161157 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6523640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b01006a0 0x7f14b0102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b5d22640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b0101060 0x7f14b01027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 -- 172.21.15.49:0/987161157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b0101060 msgr2=0x7f14b01027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b0101060 0x7f14b01027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 -- 172.21.15.49:0/987161157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b01006a0 msgr2=0x7f14b0102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b01006a0 0x7f14b0102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 -- 172.21.15.49:0/987161157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b01a6570 con 0x7f14b01066b0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14b6d24640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f14a80096f0 tx=0x7f14a80336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.496+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8043020 con 0x7f14b01066b0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.497+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a8003e50 con 0x7f14b01066b0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.497+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b01a6800 con 0x7f14b01066b0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.497+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8034690 con 0x7f14b01066b0 2024-03-16T08:22:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.497+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01a6d50 con 0x7f14b01066b0 2024-03-16T08:22:28.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.498+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b0068a20 con 0x7f14b01066b0 2024-03-16T08:22:28.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.499+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f14a8046020 con 0x7f14b01066b0 2024-03-16T08:22:28.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.500+0000 7f14a77fe640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.500+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f14a80f16a0 con 0x7f14b01066b0 2024-03-16T08:22:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.500+0000 7f14b6523640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.501+0000 7f14b6523640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 0x7f149c0a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009b50 tx=0x7f14ac005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.504+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f14a804d020 con 0x7f14b01066b0 2024-03-16T08:22:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:28 smithi049 ceph-mon[31942]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.726+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b0109c20 con 0x7f14b01066b0 2024-03-16T08:22:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.727+0000 7f14a77fe640 1 -- 172.21.15.49:0/987161157 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f14a80be520 con 0x7f14b01066b0 2024-03-16T08:22:28.729 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:22:28.729 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:22:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 msgr2=0x7f149c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 0x7f149c0a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009b50 tx=0x7f14ac005f90 comp rx=0 tx=0).stop 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 msgr2=0x7f14b0102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f14a80096f0 tx=0x7f14a80336b0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 shutdown_connections 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b01066b0 0x7f14b0102cf0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14b0101060 0x7f14b01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 --2- 172.21.15.49:0/987161157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b01006a0 0x7f14b0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.729+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 >> 172.21.15.49:0/987161157 conn(0x7f14b0077780 msgr2=0x7f14b006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.730+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 shutdown_connections 2024-03-16T08:22:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.730+0000 7f14b8787640 1 -- 172.21.15.49:0/987161157 wait complete. 2024-03-16T08:22:28.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 -- 172.21.15.49:0/732455276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 msgr2=0x7fa7b4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/732455276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4101c00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009930 tx=0x7fa7a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 -- 172.21.15.49:0/732455276 shutdown_connections 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/732455276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4102140 0x7fa7b410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/732455276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4101c00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/732455276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4107790 0x7fa7b4107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 -- 172.21.15.49:0/732455276 >> 172.21.15.49:0/732455276 conn(0x7fa7b40fd210 msgr2=0x7fa7b40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 -- 172.21.15.49:0/732455276 shutdown_connections 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.852+0000 7fa7bc47e640 1 -- 172.21.15.49:0/732455276 wait complete. 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 Processor -- start 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 -- start start 2024-03-16T08:22:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 0x7fa7b4074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 0x7fa7b4074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4111ba0 con 0x7fa7b4101780 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4111a20 con 0x7fa7b4102140 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7bc47e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4111d20 con 0x7fa7b4107790 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35684/0 (socket says 172.21.15.49:35684) 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 -- 172.21.15.49:0/4278996036 learned_addr learned my addr 172.21.15.49:0/4278996036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 0x7fa7b4074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 0x7fa7b4074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 -- 172.21.15.49:0/4278996036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 msgr2=0x7fa7b4074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 0x7fa7b4074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 -- 172.21.15.49:0/4278996036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 msgr2=0x7fa7b4074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 0x7fa7b4074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7ba21a640 1 -- 172.21.15.49:0/4278996036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b41a65e0 con 0x7fa7b4101780 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 0x7fa7b4074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.853+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 0x7fa7b4074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7ba21a640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ea60 tx=0x7fa7ac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7ab7fe640 1 -- 172.21.15.49:0/4278996036 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00cde0 con 0x7fa7b4101780 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7ab7fe640 1 -- 172.21.15.49:0/4278996036 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac0035f0 con 0x7fa7b4101780 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41a68d0 con 0x7fa7b4101780 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7ab7fe640 1 -- 172.21.15.49:0/4278996036 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac010670 con 0x7fa7b4101780 2024-03-16T08:22:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.854+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a6df0 con 0x7fa7b4101780 2024-03-16T08:22:28.864 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:28 smithi129 ceph-mon[36561]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.856+0000 7fa7ab7fe640 1 -- 172.21.15.49:0/4278996036 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa7ac0037a0 con 0x7fa7b4101780 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 Processor -- start 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 -- start start 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed77ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7ab7fe640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed786120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557aed6de490 con 0x557aed77a830 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557aed686c90 con 0x557aed77b150 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7bf066740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557aed6da530 con 0x557aed783d30 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7b9a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7ba21a640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7baa1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed77ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7b9a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42460/0 (socket says 172.21.15.49:42460) 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7b9a19640 1 -- 172.21.15.49:0/2680916207 learned_addr learned my addr 172.21.15.49:0/2680916207 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7ab7fe640 1 -- 172.21.15.49:0/4278996036 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa7ac0cf220 con 0x7fa7b4101780 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.857+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed786120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 -- 172.21.15.49:0/2680916207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 msgr2=0x557aed786120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed786120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 -- 172.21.15.49:0/2680916207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 msgr2=0x557aed77ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed77ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 -- 172.21.15.49:0/2680916207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aed741500 con 0x557aed77b150 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7ba21a640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ea30 tx=0x7fa7ac007b90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000c00 tx=0x7fa7a4005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=729cd28be1143e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7a8bfe640 1 -- 172.21.15.49:0/2680916207 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a402ca80 con 0x557aed77b150 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7a8bfe640 1 -- 172.21.15.49:0/2680916207 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a402cc20 con 0x557aed77b150 2024-03-16T08:22:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.858+0000 7fa7a8bfe640 1 -- 172.21.15.49:0/2680916207 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004920 con 0x557aed77b150 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- 172.21.15.49:0/2680916207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 msgr2=0x557aed7837f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000c00 tx=0x7fa7a4005df0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- 172.21.15.49:0/2680916207 shutdown_connections 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed786120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed7837f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2680916207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed77ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- 172.21.15.49:0/2680916207 >> 172.21.15.49:0/2680916207 conn(0x557aed774f60 msgr2=0x557aed775340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- 172.21.15.49:0/2680916207 shutdown_connections 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- 172.21.15.49:0/2680916207 wait complete. 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 -- start start 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 Processor -- start 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77a830 0x557aed780d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.859+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed781780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557aed786ac0 con 0x557aed77b150 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557aed69a230 con 0x557aed77a830 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557aed786c40 con 0x557aed783d30 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 10 client.0 _ll_register_callbacks cb 0x557aed6d6010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35694/0 (socket says 172.21.15.49:35694) 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 -- 172.21.15.49:0/73103704 learned_addr learned my addr 172.21.15.49:0/73103704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77a830 0x557aed780d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed781780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 -- 172.21.15.49:0/73103704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 msgr2=0x557aed781780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed781780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 -- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77a830 msgr2=0x557aed780d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77a830 0x557aed780d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 -- 172.21.15.49:0/73103704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aed6a9100 con 0x557aed77b150 2024-03-16T08:22:28.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402cdf0 tx=0x7fa7a402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f441499c038de7c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004240 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa7a40043e0 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a402cc20 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa7a40035f0 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 10 client.16845 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.860+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557aed3f23f0 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a402cc20 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa7a4046820 con 0x557aed77b150 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 10 client.16845 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 10 client.16845 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 20 client.16845 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed793eb0 0x557aed7962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 4 client.16845 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aed6eb9d0 con 0x557aed793eb0 2024-03-16T08:22:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed793eb0 0x557aed7962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed793eb0 0x557aed7962a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c886d06bf3a2e166 server_cookie=a876c141773b1b6d in_seq=0 out_seq=0 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.861+0000 7fa79dffb640 10 client.16845 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa79dffb640 1 -- 172.21.15.49:0/73103704 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557aed6eb9d0 con 0x557aed793eb0 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa79dffb640 10 client.16845 handle_command_reply: tid=0 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 client.16845 shutdown 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 msgr2=0x557aed781240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402cdf0 tx=0x7fa7a402ce20 comp rx=0 tx=0).stop 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 shutdown_connections 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed793eb0 0x557aed7962a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed781780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa79dffb640 0 client.16845 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77b150 0x557aed781240 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 --2- 172.21.15.49:0/73103704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77a830 0x557aed780d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 >> 172.21.15.49:0/73103704 conn(0x557aed774f60 msgr2=0x557aed776cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 shutdown_connections 2024-03-16T08:22:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 1 -- 172.21.15.49:0/73103704 wait complete. 2024-03-16T08:22:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.862+0000 7fa7bf066740 20 client.16845 trim_cache size 0 max 16384 2024-03-16T08:22:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 Processor -- start 2024-03-16T08:22:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 -- start start 2024-03-16T08:22:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed78f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed79b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41b1f70 con 0x557aed77a830 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41af840 con 0x557aed77b150 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7bf066740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41ad670 con 0x557aed783d30 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35706/0 (socket says 172.21.15.49:35706) 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 -- 172.21.15.49:0/2177811365 learned_addr learned my addr 172.21.15.49:0/2177811365 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed78f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed79b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 -- 172.21.15.49:0/2177811365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 msgr2=0x557aed79b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed79b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 -- 172.21.15.49:0/2177811365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 msgr2=0x557aed78f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed78f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 -- 172.21.15.49:0/2177811365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aed4bc7d0 con 0x557aed77a830 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.876+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009fd0 tx=0x7fa7b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a0bf3c0c54bd45 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa79ffff640 1 -- 172.21.15.49:0/2177811365 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b002a020 con 0x557aed77a830 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa79ffff640 1 -- 172.21.15.49:0/2177811365 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b0024450 con 0x557aed77a830 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa79ffff640 1 -- 172.21.15.49:0/2177811365 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0005650 con 0x557aed77a830 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- 172.21.15.49:0/2177811365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 msgr2=0x557aed7833c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009fd0 tx=0x7fa7b0004a40 comp rx=0 tx=0).stop 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- 172.21.15.49:0/2177811365 shutdown_connections 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed783d30 0x557aed79b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed78f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 --2- 172.21.15.49:0/2177811365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed77a830 0x557aed7833c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- 172.21.15.49:0/2177811365 >> 172.21.15.49:0/2177811365 conn(0x557aed774f60 msgr2=0x557aed780cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- 172.21.15.49:0/2177811365 shutdown_connections 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- 172.21.15.49:0/2177811365 wait complete. 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.877+0000 7fa7bf066740 1 -- start start 2024-03-16T08:22:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 Processor -- start 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed77a830 0x557aed787880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed787dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41add00 con 0x557aed783d30 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41af170 con 0x557aed77b150 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41ae3d0 con 0x557aed77a830 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7bf066740 10 client.0 _ll_register_callbacks cb 0x7fa7b0009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7baa1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed787dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7baa1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35708/0 (socket says 172.21.15.49:35708) 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7baa1b640 1 -- 172.21.15.49:0/447112418 learned_addr learned my addr 172.21.15.49:0/447112418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed77a830 0x557aed787880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 -- 172.21.15.49:0/447112418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed77a830 msgr2=0x557aed787880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed77a830 0x557aed787880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 -- 172.21.15.49:0/447112418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 msgr2=0x557aed788300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7b9a19640 1 -- 172.21.15.49:0/447112418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aed7f8f70 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.878+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7b9a19640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed787dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005c40 tx=0x7fa7a40466a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a588b877cfed3007 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a405b650 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa7a4034440 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b0024450 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa7a402c5d0 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 10 client.26732 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557aed7fa250 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b0024450 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa7a40646d0 con 0x557aed77b150 2024-03-16T08:22:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 10 client.26732 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 10 client.26732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 20 client.26732 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed7fa9c0 0x557aed7fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 4 client.26732 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557aed6e5d20 con 0x557aed7fa9c0 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.879+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed7fa9c0 0x557aed7fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa79dffb640 10 client.26732 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7baa1b640 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed7fa9c0 0x557aed7fcde0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f45c235748bbca95 server_cookie=fe84d96b8dacf46e in_seq=0 out_seq=0 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa79dffb640 1 -- 172.21.15.49:0/447112418 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557aed6e5d20 con 0x557aed7fa9c0 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa79dffb640 10 client.26732 handle_command_reply: tid=0 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 client.26732 shutdown 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 msgr2=0x557aed787dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed787dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005c40 tx=0x7fa7a40466a0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 shutdown_connections 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x557aed7fa9c0 0x557aed7fcde0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aed783d30 0x557aed788300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557aed77b150 0x557aed787dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 --2- 172.21.15.49:0/447112418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557aed77a830 0x557aed787880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa79dffb640 0 client.26732 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 >> 172.21.15.49:0/447112418 conn(0x557aed774f60 msgr2=0x557aed781530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 shutdown_connections 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 1 -- 172.21.15.49:0/447112418 wait complete. 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.880+0000 7fa7bf066740 20 client.26732 trim_cache size 0 max 16384 2024-03-16T08:22:28.888 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 msgr2=0x7fa7a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ea30 tx=0x7fa7ac007b90 comp rx=0 tx=0).stop 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 msgr2=0x7fa7b4073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ea60 tx=0x7fa7ac00ef20 comp rx=0 tx=0).stop 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 shutdown_connections 2024-03-16T08:22:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4107790 0x7fa7b4074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4102140 0x7fa7b4074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 --2- 172.21.15.49:0/4278996036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4101780 0x7fa7b4073ae0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 >> 172.21.15.49:0/4278996036 conn(0x7fa7b40fd210 msgr2=0x7fa7b40fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 shutdown_connections 2024-03-16T08:22:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:28.881+0000 7fa7bc47e640 1 -- 172.21.15.49:0/4278996036 wait complete. 2024-03-16T08:22:28.893 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:22:28.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:28 smithi154 ceph-mon[36471]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 -- 172.21.15.49:0/3710997950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 msgr2=0x7fc66010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 --2- 172.21.15.49:0/3710997950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc66010b850 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc650009900 tx=0x7fc65002bcd0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 -- 172.21.15.49:0/3710997950 shutdown_connections 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 --2- 172.21.15.49:0/3710997950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 0x7fc66010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 --2- 172.21.15.49:0/3710997950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc66010b850 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 --2- 172.21.15.49:0/3710997950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 0x7fc660102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.001+0000 7fc667fab640 1 -- 172.21.15.49:0/3710997950 >> 172.21.15.49:0/3710997950 conn(0x7fc660077840 msgr2=0x7fc660077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- 172.21.15.49:0/3710997950 shutdown_connections 2024-03-16T08:22:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- 172.21.15.49:0/3710997950 wait complete. 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 Processor -- start 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- start start 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 0x7fc660198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 0x7fc66019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111bf0 con 0x7fc660103370 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111a70 con 0x7fc66010be20 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc667fab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111d70 con 0x7fc660102a50 2024-03-16T08:22:29.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.002+0000 7fc665546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35722/0 (socket says 172.21.15.49:35722) 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 -- 172.21.15.49:0/4020550502 learned_addr learned my addr 172.21.15.49:0/4020550502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665d47640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 0x7fc660198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc666548640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 0x7fc66019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 -- 172.21.15.49:0/4020550502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 msgr2=0x7fc660198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 0x7fc660198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 -- 172.21.15.49:0/4020550502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 msgr2=0x7fc66019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 0x7fc66019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 -- 172.21.15.49:0/4020550502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66019d0f0 con 0x7fc660103370 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc665546640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc650005940 tx=0x7fc65002cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc656ffd640 1 -- 172.21.15.49:0/4020550502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650041070 con 0x7fc660103370 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66019d380 con 0x7fc660103370 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc656ffd640 1 -- 172.21.15.49:0/4020550502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc650004450 con 0x7fc660103370 2024-03-16T08:22:29.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc656ffd640 1 -- 172.21.15.49:0/4020550502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650003d80 con 0x7fc660103370 2024-03-16T08:22:29.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.003+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6601a4c70 con 0x7fc660103370 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.005+0000 7fc656ffd640 1 -- 172.21.15.49:0/4020550502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc650044070 con 0x7fc660103370 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 Processor -- start 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 -- start start 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4dda0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 0x562c4ddb0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c4dd04490 con 0x562c4dda0e80 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c4dcacc90 con 0x562c4ddae140 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562c4dd00530 con 0x562c4dda0560 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc656ffd640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 0x7fc64c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc665d47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc666548640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4dda0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc665546640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 0x562c4ddb0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc665d47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35724/0 (socket says 172.21.15.49:35724) 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc665d47640 1 -- 172.21.15.49:0/4045096973 learned_addr learned my addr 172.21.15.49:0/4045096973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc666548640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 0x7fc64c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.006+0000 7fc656ffd640 1 -- 172.21.15.49:0/4020550502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc6500f07b0 con 0x7fc660103370 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 -- 172.21.15.49:0/4045096973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 msgr2=0x562c4ddb0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 0x562c4ddb0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 -- 172.21.15.49:0/4045096973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 msgr2=0x562c4ddadc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 -- 172.21.15.49:0/4045096973 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4dd67500 con 0x562c4dda0560 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc665d47640 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc665546640 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 0x562c4ddb0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 0x7fc64c0a4e30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc65800fc90 tx=0x7fc658018000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc666548640 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4dda0940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc658011cb0 tx=0x7fc65800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b17c2d8bdc8afa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc64bfff640 1 -- 172.21.15.49:0/4045096973 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658011e60 con 0x562c4dda0560 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc64bfff640 1 -- 172.21.15.49:0/4045096973 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc658009cc0 con 0x562c4dda0560 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.007+0000 7fc64bfff640 1 -- 172.21.15.49:0/4045096973 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658005830 con 0x562c4dda0560 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- 172.21.15.49:0/4045096973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 msgr2=0x562c4dda0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4dda0940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc658011cb0 tx=0x7fc65800f790 comp rx=0 tx=0).stop 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- 172.21.15.49:0/4045096973 shutdown_connections 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4ddae140 0x562c4ddb0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddadc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/4045096973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4dda0940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- 172.21.15.49:0/4045096973 >> 172.21.15.49:0/4045096973 conn(0x562c4dd9af60 msgr2=0x562c4dd9b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- 172.21.15.49:0/4045096973 shutdown_connections 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- 172.21.15.49:0/4045096973 wait complete. 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 -- start start 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.008+0000 7fc66ab8a740 1 Processor -- start 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4ddabc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c4ddb0e40 con 0x562c4dda0560 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c4dcc0230 con 0x562c4dda0e80 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562c4ddb0fc0 con 0x562c4ddae140 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc66ab8a740 10 client.0 _ll_register_callbacks cb 0x562c4dcfc010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35730/0 (socket says 172.21.15.49:35730) 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 -- 172.21.15.49:0/605934169 learned_addr learned my addr 172.21.15.49:0/605934169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665546640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc666548640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4ddabc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 -- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 msgr2=0x562c4ddabc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4ddabc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 -- 172.21.15.49:0/605934169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 msgr2=0x562c4ddab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 -- 172.21.15.49:0/605934169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4dccf100 con 0x562c4dda0560 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665546640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc665d47640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc65c012720 tx=0x7fc65c012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95643a3b7904b4d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:29.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.009+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c004050 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc658009cc0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc65c0041f0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc65c0043f0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 10 client.16863 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562c4da183f0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc658009cc0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc65c003bf0 con 0x562c4dda0560 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 10 client.16863 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 20 client.16863 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4ddbe470 0x562c4ddc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 4 client.16863 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c4dd119d0 con 0x562c4ddbe470 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.010+0000 7fc665546640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4ddbe470 0x562c4ddc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.011+0000 7fc6497fa640 10 client.16863 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.011+0000 7fc665546640 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4ddbe470 0x562c4ddc08a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c48d45781b65d6ca server_cookie=64b26ee8ea56b83d in_seq=0 out_seq=0 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.011+0000 7fc6497fa640 1 -- 172.21.15.49:0/605934169 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c4dd119d0 con 0x562c4ddbe470 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.011+0000 7fc6497fa640 10 client.16863 handle_command_reply: tid=0 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.011+0000 7fc66ab8a740 1 client.16863 shutdown 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 msgr2=0x562c4ddab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc65c012720 tx=0x7fc65c012b20 comp rx=0 tx=0).stop 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 shutdown_connections 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4ddbe470 0x562c4ddc08a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4ddabc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/605934169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0560 0x562c4ddab1e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc6497fa640 0 client.16863 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 >> 172.21.15.49:0/605934169 conn(0x562c4dd9af60 msgr2=0x562c4dd9ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 shutdown_connections 2024-03-16T08:22:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 1 -- 172.21.15.49:0/605934169 wait complete. 2024-03-16T08:22:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.012+0000 7fc66ab8a740 20 client.16863 trim_cache size 0 max 16384 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 Processor -- start 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 -- start start 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0560 0x562c4ddad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4dd99e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ada50 con 0x562c4dda0e80 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ab320 con 0x562c4dda0560 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601a9150 con 0x562c4ddae140 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35746/0 (socket says 172.21.15.49:35746) 2024-03-16T08:22:29.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 -- 172.21.15.49:0/3044595384 learned_addr learned my addr 172.21.15.49:0/3044595384 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc666548640 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4dd99e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665d47640 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0560 0x562c4ddad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 -- 172.21.15.49:0/3044595384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 msgr2=0x562c4dd99e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4dd99e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 -- 172.21.15.49:0/3044595384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0560 msgr2=0x562c4ddad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0560 0x562c4ddad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.025+0000 7fc665546640 1 -- 172.21.15.49:0/3044595384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4dae27d0 con 0x562c4dda0e80 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc665546640 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b9480 tx=0x7fc650087d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dbf5cc145d1d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc64b7fe640 1 -- 172.21.15.49:0/3044595384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65004b040 con 0x562c4dda0e80 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc64b7fe640 1 -- 172.21.15.49:0/3044595384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc650004c80 con 0x562c4dda0e80 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc64b7fe640 1 -- 172.21.15.49:0/3044595384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500c10c0 con 0x562c4dda0e80 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 -- 172.21.15.49:0/3044595384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 msgr2=0x562c4ddb9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b9480 tx=0x7fc650087d90 comp rx=0 tx=0).stop 2024-03-16T08:22:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 -- 172.21.15.49:0/3044595384 shutdown_connections 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4ddae140 0x562c4dd99e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4dda0e80 0x562c4ddb9c20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/3044595384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0560 0x562c4ddad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 -- 172.21.15.49:0/3044595384 >> 172.21.15.49:0/3044595384 conn(0x562c4dd9af60 msgr2=0x562c4dd99750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 -- 172.21.15.49:0/3044595384 shutdown_connections 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.026+0000 7fc66ab8a740 1 -- 172.21.15.49:0/3044595384 wait complete. 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 -- start start 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 Processor -- start 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddb1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4ddae140 0x562c4ddb1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601a97e0 con 0x562c4ddae140 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601aac50 con 0x562c4dda0e80 2024-03-16T08:22:29.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601a9eb0 con 0x562c4dda0560 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc66ab8a740 10 client.0 _ll_register_callbacks cb 0x7fc65c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc665d47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc666548640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4ddae140 0x562c4ddb1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc665d47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36412/0 (socket says 172.21.15.49:36412) 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc665546640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddb1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.027+0000 7fc665d47640 1 -- 172.21.15.49:0/1234629347 learned_addr learned my addr 172.21.15.49:0/1234629347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 -- 172.21.15.49:0/1234629347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 msgr2=0x562c4ddb1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddb1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 -- 172.21.15.49:0/1234629347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4ddae140 msgr2=0x562c4ddb1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4ddae140 0x562c4ddb1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 -- 172.21.15.49:0/1234629347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4ddb2780 con 0x562c4dda0560 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665546640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddb1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc665d47640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc65c012410 tx=0x7fc65c003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17946c70446543dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c02c070 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc650004c80 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fc65c026ae0 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc65c026ce0 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc66ab8a740 10 client.26752 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562c4de248a0 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc650004c80 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.028+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc65c005570 con 0x562c4dda0560 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 10 client.26752 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 10 client.26752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 20 client.26752 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4de25070 0x562c4de27490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 4 client.26752 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c4dd0bd20 con 0x562c4de25070 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc665546640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4de25070 0x562c4de27490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc6497fa640 10 client.26752 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc665546640 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4de25070 0x562c4de27490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ab0da59d6dce1b3 server_cookie=f3d93b71a63a0f92 in_seq=0 out_seq=0 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc6497fa640 1 -- 172.21.15.49:0/1234629347 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c4dd0bd20 con 0x562c4de25070 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc6497fa640 10 client.26752 handle_command_reply: tid=0 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 client.26752 shutdown 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 msgr2=0x562c4ddb6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc65c012410 tx=0x7fc65c003f90 comp rx=0 tx=0).stop 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 shutdown_connections 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x562c4de25070 0x562c4de27490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c4ddae140 0x562c4ddb1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c4dda0e80 0x562c4ddb1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc6497fa640 0 client.26752 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 --2- 172.21.15.49:0/1234629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c4dda0560 0x562c4ddb6600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.029+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 >> 172.21.15.49:0/1234629347 conn(0x562c4dd9af60 msgr2=0x562c4ddab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 shutdown_connections 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc66ab8a740 1 -- 172.21.15.49:0/1234629347 wait complete. 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc66ab8a740 20 client.26752 trim_cache size 0 max 16384 2024-03-16T08:22:29.038 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 msgr2=0x7fc64c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 0x7fc64c0a4e30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc65800fc90 tx=0x7fc658018000 comp rx=0 tx=0).stop 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 msgr2=0x7fc660198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc650005940 tx=0x7fc65002cbd0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 shutdown_connections 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc64c0a2970 0x7fc64c0a4e30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc66010be20 0x7fc66019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc660103370 0x7fc660198690 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 --2- 172.21.15.49:0/4020550502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc660102a50 0x7fc660198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 >> 172.21.15.49:0/4020550502 conn(0x7fc660077840 msgr2=0x7fc6600fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 shutdown_connections 2024-03-16T08:22:29.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.030+0000 7fc667fab640 1 -- 172.21.15.49:0/4020550502 wait complete. 2024-03-16T08:22:29.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/1912929622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 msgr2=0x7fd8a0101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/1912929622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a0101c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd894009890 tx=0x7fd89402bb70 comp rx=0 tx=0).stop 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/1912929622 shutdown_connections 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/1912929622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 0x7fd8a010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/1912929622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a0101c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/1912929622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a0107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/1912929622 >> 172.21.15.49:0/1912929622 conn(0x7fd8a00fd210 msgr2=0x7fd8a00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/1912929622 shutdown_connections 2024-03-16T08:22:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.152+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/1912929622 wait complete. 2024-03-16T08:22:29.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 Processor -- start 2024-03-16T08:22:29.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 -- start start 2024-03-16T08:22:29.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 0x7fd8a0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0111ba0 con 0x7fd8a0107790 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0111a20 con 0x7fd8a0102140 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd8a4dd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0111d20 con 0x7fd8a0101780 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd89ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd89ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35762/0 (socket says 172.21.15.49:35762) 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd89ed9d640 1 -- 172.21.15.49:0/72269893 learned_addr learned my addr 172.21.15.49:0/72269893 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 0x7fd8a0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.153+0000 7fd89e59c640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 -- 172.21.15.49:0/72269893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 msgr2=0x7fd8a01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 -- 172.21.15.49:0/72269893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 msgr2=0x7fd8a0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 0x7fd8a0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 -- 172.21.15.49:0/72269893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a019d0d0 con 0x7fd8a0107790 2024-03-16T08:22:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd89ed9d640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd89800eb10 tx=0x7fd89800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd88f7fe640 1 -- 172.21.15.49:0/72269893 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898004b20 con 0x7fd8a0107790 2024-03-16T08:22:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a019d3c0 con 0x7fd8a0107790 2024-03-16T08:22:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd88f7fe640 1 -- 172.21.15.49:0/72269893 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd898004cc0 con 0x7fd8a0107790 2024-03-16T08:22:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd88f7fe640 1 -- 172.21.15.49:0/72269893 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898010760 con 0x7fd8a0107790 2024-03-16T08:22:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.154+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a4c50 con 0x7fd8a0107790 2024-03-16T08:22:29.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd88f7fe640 1 -- 172.21.15.49:0/72269893 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd898005020 con 0x7fd8a0107790 2024-03-16T08:22:29.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 Processor -- start 2024-03-16T08:22:29.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 -- start start 2024-03-16T08:22:29.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f16c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561699e7a490 con 0x561699f1fd30 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561699e22c90 con 0x561699f16830 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561699e76530 con 0x561699f17150 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd88f7fe640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 0x7fd8880a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd89e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35772/0 (socket says 172.21.15.49:35772) 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3171705088 learned_addr learned my addr 172.21.15.49:0/3171705088 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.157+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f16c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89ed9d640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 0x7fd8880a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd88f7fe640 1 -- 172.21.15.49:0/72269893 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd8980cf4e0 con 0x7fd8a0107790 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3171705088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 msgr2=0x561699f1f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3171705088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 msgr2=0x561699f16c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f16c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3171705088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561699edd500 con 0x561699f1fd30 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89e59c640 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd894005e00 tx=0x7fd89402bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91a368b4b80a66b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd89ed9d640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 0x7fd8880a4e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd898098560 tx=0x7fd898091020 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd88cff9640 1 -- 172.21.15.49:0/3171705088 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894039040 con 0x561699f1fd30 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd88cff9640 1 -- 172.21.15.49:0/3171705088 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894034cb0 con 0x561699f1fd30 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.158+0000 7fd88cff9640 1 -- 172.21.15.49:0/3171705088 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894004470 con 0x561699f1fd30 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3171705088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 msgr2=0x561699f22120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd894005e00 tx=0x7fd89402bb70 comp rx=0 tx=0).stop 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3171705088 shutdown_connections 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f22120 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3171705088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f16c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3171705088 >> 172.21.15.49:0/3171705088 conn(0x561699f10f60 msgr2=0x561699f11340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3171705088 shutdown_connections 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3171705088 wait complete. 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 -- start start 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.159+0000 7fd8a79c4740 1 Processor -- start 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561699f22a60 con 0x561699f1fd30 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561699e36230 con 0x561699f16830 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561699f22be0 con 0x561699f17150 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd8a79c4740 10 client.0 _ll_register_callbacks cb 0x561699e72010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35780/0 (socket says 172.21.15.49:35780) 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3037949710 learned_addr learned my addr 172.21.15.49:0/3037949710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3037949710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 msgr2=0x561699f1d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 msgr2=0x561699f1cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 -- 172.21.15.49:0/3037949710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561699e45100 con 0x561699f1fd30 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89e59c640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd894009860 tx=0x7fd894005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c662c9cae5d45004 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.160+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894039040 con 0x561699f1fd30 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd894034640 con 0x561699f1fd30 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd894034cb0 con 0x561699f1fd30 2024-03-16T08:22:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd894004490 con 0x561699f1fd30 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 10 client.16881 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561699b8e3f0 con 0x561699f1fd30 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894034cb0 con 0x561699f1fd30 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd89402c910 con 0x561699f1fd30 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 10 client.16881 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 20 client.16881 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f2fee0 0x561699f322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 4 client.16881 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561699e879d0 con 0x561699f2fee0 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.161+0000 7fd89e59c640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f2fee0 0x561699f322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.162+0000 7fd885ffb640 10 client.16881 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.162+0000 7fd89e59c640 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f2fee0 0x561699f322d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9232cd5ec1c95bf8 server_cookie=1d33111c928528fd in_seq=0 out_seq=0 2024-03-16T08:22:29.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd885ffb640 1 -- 172.21.15.49:0/3037949710 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561699e879d0 con 0x561699f2fee0 2024-03-16T08:22:29.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd885ffb640 10 client.16881 handle_command_reply: tid=0 2024-03-16T08:22:29.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 client.16881 shutdown 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 msgr2=0x561699f1d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd894009860 tx=0x7fd894005850 comp rx=0 tx=0).stop 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 shutdown_connections 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f2fee0 0x561699f322d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d810 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f1d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd885ffb640 0 client.16881 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/3037949710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 >> 172.21.15.49:0/3037949710 conn(0x561699f10f60 msgr2=0x561699f12cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 shutdown_connections 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 1 -- 172.21.15.49:0/3037949710 wait complete. 2024-03-16T08:22:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.163+0000 7fd8a79c4740 20 client.16881 trim_cache size 0 max 16384 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.176+0000 7fd8a79c4740 1 Processor -- start 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 -- start start 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01afd00 con 0x561699f1fd30 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01ad5d0 con 0x561699f16830 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01ab400 con 0x561699f17150 2024-03-16T08:22:29.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35786/0 (socket says 172.21.15.49:35786) 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 -- 172.21.15.49:0/1939662695 learned_addr learned my addr 172.21.15.49:0/1939662695 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89e59c640 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 -- 172.21.15.49:0/1939662695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 msgr2=0x561699f352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 -- 172.21.15.49:0/1939662695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 msgr2=0x561699f1f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 -- 172.21.15.49:0/1939662695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561699f16f80 con 0x561699f1fd30 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.177+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd894004a40 tx=0x7fd894005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa1b22f70ed33cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd887fff640 1 -- 172.21.15.49:0/1939662695 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894039070 con 0x561699f1fd30 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd887fff640 1 -- 172.21.15.49:0/1939662695 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894061070 con 0x561699f1fd30 2024-03-16T08:22:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd887fff640 1 -- 172.21.15.49:0/1939662695 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89402c750 con 0x561699f1fd30 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 -- 172.21.15.49:0/1939662695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 msgr2=0x561699f1d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd894004a40 tx=0x7fd894005c90 comp rx=0 tx=0).stop 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 -- 172.21.15.49:0/1939662695 shutdown_connections 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f1fd30 0x561699f1d460 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f17150 0x561699f352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.178+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/1939662695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f16830 0x561699f1f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- 172.21.15.49:0/1939662695 >> 172.21.15.49:0/1939662695 conn(0x561699f10f60 msgr2=0x561699f1cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- 172.21.15.49:0/1939662695 shutdown_connections 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- 172.21.15.49:0/1939662695 wait complete. 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- start start 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 Processor -- start 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 0x561699f32af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f1fd30 0x561699f33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01aba90 con 0x561699f16830 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01acf00 con 0x561699f17150 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01ac160 con 0x561699f1fd30 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.179+0000 7fd8a79c4740 10 client.0 _ll_register_callbacks cb 0x7fd8980093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 0x561699f32af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35792/0 (socket says 172.21.15.49:35792) 2024-03-16T08:22:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89dd9b640 1 -- 172.21.15.49:0/2777776380 learned_addr learned my addr 172.21.15.49:0/2777776380 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f1fd30 0x561699f33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 -- 172.21.15.49:0/2777776380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f1fd30 msgr2=0x561699f33230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f1fd30 0x561699f33230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 -- 172.21.15.49:0/2777776380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 msgr2=0x561699f35760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 -- 172.21.15.49:0/2777776380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561699c587d0 con 0x561699f17150 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd89e59c640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 0x561699f32af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x561699f22c50 tx=0x7fd890012ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d03ab33c3ad85b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890024070 con 0x561699f17150 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fd890004220 con 0x561699f17150 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd894061070 con 0x561699f17150 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.180+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd89002c020 con 0x561699f17150 2024-03-16T08:22:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 10 client.26768 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561699f95ed0 con 0x561699f17150 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894061070 con 0x561699f17150 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd890005470 con 0x561699f17150 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 10 client.26768 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 10 client.26768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 20 client.26768 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f96600 0x561699f98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 4 client.26768 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561699e81d20 con 0x561699f96600 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f96600 0x561699f98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd89dd9b640 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f96600 0x561699f98a20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab960d97e2577322 server_cookie=15ebffbe1a17441 in_seq=0 out_seq=0 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.181+0000 7fd885ffb640 10 client.26768 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd885ffb640 1 -- 172.21.15.49:0/2777776380 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561699e81d20 con 0x561699f96600 2024-03-16T08:22:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd885ffb640 10 client.26768 handle_command_reply: tid=0 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 client.26768 shutdown 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 msgr2=0x561699f32af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 0x561699f32af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x561699f22c50 tx=0x7fd890012ae0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 shutdown_connections 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561699f96600 0x561699f98a20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561699f1fd30 0x561699f33230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561699f17150 0x561699f32af0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd885ffb640 0 client.26768 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 --2- 172.21.15.49:0/2777776380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561699f16830 0x561699f35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 >> 172.21.15.49:0/2777776380 conn(0x561699f10f60 msgr2=0x561699f2b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 shutdown_connections 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 1 -- 172.21.15.49:0/2777776380 wait complete. 2024-03-16T08:22:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a79c4740 20 client.26768 trim_cache size 0 max 16384 2024-03-16T08:22:29.190 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:22:29.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 msgr2=0x7fd8880a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 0x7fd8880a4e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd898098560 tx=0x7fd898091020 comp rx=0 tx=0).stop 2024-03-16T08:22:29.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 msgr2=0x7fd8a019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.182+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd89800eb10 tx=0x7fd89800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 shutdown_connections 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd8880a2970 0x7fd8880a4e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a0107790 0x7fd8a019c9a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8a0102140 0x7fd8a0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 --2- 172.21.15.49:0/72269893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a0101780 0x7fd8a01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 >> 172.21.15.49:0/72269893 conn(0x7fd8a00fd210 msgr2=0x7fd8a00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 shutdown_connections 2024-03-16T08:22:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:29.183+0000 7fd8a4dd4640 1 -- 172.21.15.49:0/72269893 wait complete. 2024-03-16T08:22:29.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:29 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/987161157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:22:29.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:29 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/987161157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:22:29.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:29 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/987161157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:22:30.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:22:30.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:22:30.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 -- 172.21.15.49:0/3997507259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb01010c0 msgr2=0x7f5bb010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/3997507259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb01010c0 0x7f5bb010d5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009ca0 tx=0x7f5ba802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 -- 172.21.15.49:0/3997507259 shutdown_connections 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/3997507259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb01010c0 0x7f5bb010d5d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/3997507259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 0x7f5bb0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/3997507259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb0106710 0x7f5bb0106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 -- 172.21.15.49:0/3997507259 >> 172.21.15.49:0/3997507259 conn(0x7f5bb0077840 msgr2=0x7f5bb0077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 -- 172.21.15.49:0/3997507259 shutdown_connections 2024-03-16T08:22:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.319+0000 7f5bb6a68640 1 -- 172.21.15.49:0/3997507259 wait complete. 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 Processor -- start 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 -- start start 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 0x7f5bb0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb0106710 0x7f5bb01a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0110b20 con 0x7f5bb01010c0 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01109a0 con 0x7f5bb0106710 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb6a68640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0110ca0 con 0x7f5bb0100700 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb5005640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb0106710 0x7f5bb01a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35816/0 (socket says 172.21.15.49:35816) 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5baffff640 1 -- 172.21.15.49:0/2414720560 learned_addr learned my addr 172.21.15.49:0/2414720560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.320+0000 7f5bb4804640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 0x7f5bb0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 -- 172.21.15.49:0/2414720560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 msgr2=0x7f5bb0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 0x7f5bb0198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 -- 172.21.15.49:0/2414720560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb0106710 msgr2=0x7f5bb01a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb0106710 0x7f5bb01a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 -- 172.21.15.49:0/2414720560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb01a4480 con 0x7f5bb01010c0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5baffff640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400ea60 tx=0x7f5ba400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5badffb640 1 -- 172.21.15.49:0/2414720560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400cde0 con 0x7f5bb01010c0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb01a4650 con 0x7f5bb01010c0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5badffb640 1 -- 172.21.15.49:0/2414720560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba40035f0 con 0x7f5bb01010c0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5badffb640 1 -- 172.21.15.49:0/2414720560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4010670 con 0x7f5bb01010c0 2024-03-16T08:22:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.321+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb01a4b10 con 0x7f5bb01010c0 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.323+0000 7f5badffb640 1 -- 172.21.15.49:0/2414720560 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5ba40037a0 con 0x7f5bb01010c0 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 Processor -- start 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 -- start start 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5badffb640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 0x7f5b980a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe6a30 0x55db0bbe6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55db0bb4a0d0 con 0x55db0bbf41b0 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db0baf2c70 con 0x55db0bbe7710 2024-03-16T08:22:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb964f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55db0bb46170 con 0x55db0bbe6a30 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5badffb640 1 -- 172.21.15.49:0/2414720560 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5ba40cf070 con 0x7f5bb01010c0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb5005640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 0x7f5b980a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42614/0 (socket says 172.21.15.49:42614) 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb4804640 1 -- 172.21.15.49:0/2709911554 learned_addr learned my addr 172.21.15.49:0/2709911554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5bb5005640 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.324+0000 7f5baffff640 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe6a30 0x55db0bbe6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 -- 172.21.15.49:0/2709911554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe6a30 msgr2=0x55db0bbe6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe6a30 0x55db0bbe6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 -- 172.21.15.49:0/2709911554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 msgr2=0x55db0bbf65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 -- 172.21.15.49:0/2709911554 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db0b9267e0 con 0x55db0bbe7710 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb5005640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 0x7f5b980a4e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005e30 tx=0x7f5ba8032340 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5bb4804640 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009c70 tx=0x7f5ba0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ccb0641771e4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5b96ffd640 1 -- 172.21.15.49:0/2709911554 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba001e070 con 0x55db0bbe7710 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5b96ffd640 1 -- 172.21.15.49:0/2709911554 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba0019b10 con 0x55db0bbe7710 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.325+0000 7f5b96ffd640 1 -- 172.21.15.49:0/2709911554 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000e9b0 con 0x55db0bbe7710 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- 172.21.15.49:0/2709911554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 msgr2=0x55db0bbe6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009c70 tx=0x7f5ba0011810 comp rx=0 tx=0).stop 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- 172.21.15.49:0/2709911554 shutdown_connections 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe6a30 0x55db0bbe6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 --2- 172.21.15.49:0/2709911554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbe6460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- 172.21.15.49:0/2709911554 >> 172.21.15.49:0/2709911554 conn(0x55db0bbadc50 msgr2=0x55db0bbe1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- 172.21.15.49:0/2709911554 shutdown_connections 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- 172.21.15.49:0/2709911554 wait complete. 2024-03-16T08:22:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 -- start start 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.326+0000 7f5bb964f740 1 Processor -- start 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbf1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55db0bbf7020 con 0x55db0bbe6a30 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db0bb060d0 con 0x55db0bbe7710 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55db0bbf71a0 con 0x55db0bbf41b0 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb964f740 10 client.0 _ll_register_callbacks cb 0x55db0bacf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35836/0 (socket says 172.21.15.49:35836) 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 -- 172.21.15.49:0/1269111151 learned_addr learned my addr 172.21.15.49:0/1269111151 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5baffff640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb5005640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbf1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 -- 172.21.15.49:0/1269111151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 msgr2=0x55db0bbf1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbf1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 -- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 msgr2=0x55db0bbf16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 -- 172.21.15.49:0/1269111151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db0b986f70 con 0x55db0bbe6a30 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5baffff640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5bb4804640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0000c00 tx=0x7f5ba0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247da2f756b62920 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.327+0000 7f5b8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba001e050 con 0x55db0bbe6a30 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5ba0019840 con 0x55db0bbe6a30 2024-03-16T08:22:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba0019b10 con 0x55db0bbe6a30 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ba0033050 con 0x55db0bbe6a30 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 10 client.16899 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55db0b81e170 con 0x55db0bbe6a30 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba0019b10 con 0x55db0bbe6a30 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ba00035f0 con 0x55db0bbe6a30 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 10 client.16899 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 20 client.16899 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc04220 0x55db0bc06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 4 client.16899 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db0bb57610 con 0x55db0bc04220 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.328+0000 7f5baffff640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc04220 0x55db0bc06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.329+0000 7f5b8ffff640 10 client.16899 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.329+0000 7f5baffff640 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc04220 0x55db0bc06610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fba63938d14379 server_cookie=1d5180312e28c289 in_seq=0 out_seq=0 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.329+0000 7f5b8ffff640 1 -- 172.21.15.49:0/1269111151 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db0bb57610 con 0x55db0bc04220 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.329+0000 7f5b8ffff640 10 client.16899 handle_command_reply: tid=0 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.329+0000 7f5bb964f740 1 client.16899 shutdown 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 msgr2=0x55db0bbf1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0000c00 tx=0x7f5ba0004270 comp rx=0 tx=0).stop 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 shutdown_connections 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc04220 0x55db0bc06610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbf1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 --2- 172.21.15.49:0/1269111151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf1170 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5b8ffff640 0 client.16899 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 >> 172.21.15.49:0/1269111151 conn(0x55db0bbadc50 msgr2=0x55db0bbe2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 shutdown_connections 2024-03-16T08:22:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 1 -- 172.21.15.49:0/1269111151 wait complete. 2024-03-16T08:22:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.330+0000 7f5bb964f740 20 client.16899 trim_cache size 0 max 16384 2024-03-16T08:22:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 Processor -- start 2024-03-16T08:22:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 -- start start 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ada10 con 0x55db0bbe6a30 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ab2e0 con 0x55db0bbe7710 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb964f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01a9110 con 0x55db0bbf41b0 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35846/0 (socket says 172.21.15.49:35846) 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 -- 172.21.15.49:0/756379320 learned_addr learned my addr 172.21.15.49:0/756379320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb5005640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5baffff640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 -- 172.21.15.49:0/756379320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 msgr2=0x55db0bbffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 -- 172.21.15.49:0/756379320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 msgr2=0x55db0bbf02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 -- 172.21.15.49:0/756379320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db0bbe2050 con 0x55db0bbe6a30 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5baffff640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb5005640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.348+0000 7f5bb4804640 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00059f0 tx=0x7f5ba0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd6f488d34892d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5b967fc640 1 -- 172.21.15.49:0/756379320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba003c070 con 0x55db0bbe6a30 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5b967fc640 1 -- 172.21.15.49:0/756379320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba0015070 con 0x55db0bbe6a30 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5b967fc640 1 -- 172.21.15.49:0/756379320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba001e040 con 0x55db0bbe6a30 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 -- 172.21.15.49:0/756379320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 msgr2=0x55db0bbf38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00059f0 tx=0x7f5ba0011d10 comp rx=0 tx=0).stop 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 -- 172.21.15.49:0/756379320 shutdown_connections 2024-03-16T08:22:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbf41b0 0x55db0bbffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe7710 0x55db0bbf02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 --2- 172.21.15.49:0/756379320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbe6a30 0x55db0bbf38a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 -- 172.21.15.49:0/756379320 >> 172.21.15.49:0/756379320 conn(0x55db0bbadc50 msgr2=0x55db0bbdfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 -- 172.21.15.49:0/756379320 shutdown_connections 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.349+0000 7f5bb964f740 1 -- 172.21.15.49:0/756379320 wait complete. 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 -- start start 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 Processor -- start 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 0x55db0bbf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe7710 0x55db0bbf5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01a97a0 con 0x55db0bbf41b0 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01aac10 con 0x55db0bbe6a30 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01a9e70 con 0x55db0bbe7710 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb964f740 10 client.0 _ll_register_callbacks cb 0x7f5ba001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 0x55db0bbf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35848/0 (socket says 172.21.15.49:35848) 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 -- 172.21.15.49:0/26271304 learned_addr learned my addr 172.21.15.49:0/26271304 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb4804640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 0x55db0bbf4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42648/0 (socket says 172.21.15.49:42648) 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5baffff640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe7710 0x55db0bbf5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 -- 172.21.15.49:0/26271304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe7710 msgr2=0x55db0bbf5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe7710 0x55db0bbf5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 -- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 msgr2=0x55db0bbf4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 0x55db0bbf4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.350+0000 7f5bb5005640 1 -- 172.21.15.49:0/26271304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db0bbe5a60 con 0x55db0bbf41b0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb5005640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8036000 tx=0x7f5ba8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb6a92fe364504f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8052070 con 0x55db0bbf41b0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5ba8003970 con 0x55db0bbf41b0 2024-03-16T08:22:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba0015070 con 0x55db0bbf41b0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ba805e050 con 0x55db0bbf41b0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 10 client.16905 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55db0bc6a570 con 0x55db0bbf41b0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba0015070 con 0x55db0bbf41b0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ba8005070 con 0x55db0bbf41b0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 10 client.16905 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 20 client.16905 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc6acf0 0x55db0bc6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 4 client.16905 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.351+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55db0bb51960 con 0x55db0bc6acf0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.352+0000 7f5bb4804640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc6acf0 0x55db0bc6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.352+0000 7f5bb4804640 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc6acf0 0x55db0bc6d110 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf20b457fbbffaad server_cookie=961c749f73e9ba20 in_seq=0 out_seq=0 2024-03-16T08:22:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.352+0000 7f5b8ffff640 10 client.16905 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5b8ffff640 1 -- 172.21.15.49:0/26271304 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55db0bb51960 con 0x55db0bc6acf0 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5b8ffff640 10 client.16905 handle_command_reply: tid=0 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 client.16905 shutdown 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 msgr2=0x55db0bbf5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8036000 tx=0x7f5ba8005d80 comp rx=0 tx=0).stop 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 shutdown_connections 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55db0bc6acf0 0x55db0bc6d110 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db0bbf41b0 0x55db0bbf5690 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55db0bbe7710 0x55db0bbf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5b8ffff640 0 client.16905 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 --2- 172.21.15.49:0/26271304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db0bbe6a30 0x55db0bbf4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 >> 172.21.15.49:0/26271304 conn(0x55db0bbadc50 msgr2=0x55db0bbe2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 shutdown_connections 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 1 -- 172.21.15.49:0/26271304 wait complete. 2024-03-16T08:22:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.353+0000 7f5bb964f740 20 client.16905 trim_cache size 0 max 16384 2024-03-16T08:22:30.361 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:22:30.361 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:22:30.361 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "72628079-597c-4df2-b9b2-05955656c8d5", 2024-03-16T08:22:30.361 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:22:30.361 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 msgr2=0x7f5b980a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 0x7f5b980a4e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005e30 tx=0x7f5ba8032340 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 msgr2=0x7f5bb0199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400ea60 tx=0x7f5ba400ef20 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 shutdown_connections 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5b980a2970 0x7f5b980a4e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bb0106710 0x7f5bb01a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bb01010c0 0x7f5bb0199400 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 --2- 172.21.15.49:0/2414720560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bb0100700 0x7f5bb0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 >> 172.21.15.49:0/2414720560 conn(0x7f5bb0077840 msgr2=0x7f5bb0103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 shutdown_connections 2024-03-16T08:22:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.354+0000 7f5bb6a68640 1 -- 172.21.15.49:0/2414720560 wait complete. 2024-03-16T08:22:30.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:22:30.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:22:30.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 -- 172.21.15.49:0/2139096214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae38102a50 msgr2=0x7fae38102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 --2- 172.21.15.49:0/2139096214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae38102a50 0x7fae38102e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae28009860 tx=0x7fae2802bc20 comp rx=0 tx=0).stop 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 -- 172.21.15.49:0/2139096214 shutdown_connections 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 --2- 172.21.15.49:0/2139096214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae3810be20 0x7fae3810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 --2- 172.21.15.49:0/2139096214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae3810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 --2- 172.21.15.49:0/2139096214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae38102a50 0x7fae38102e30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 -- 172.21.15.49:0/2139096214 >> 172.21.15.49:0/2139096214 conn(0x7fae38077840 msgr2=0x7fae38077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.475+0000 7fae3d13b640 1 -- 172.21.15.49:0/2139096214 shutdown_connections 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.476+0000 7fae3d13b640 1 -- 172.21.15.49:0/2139096214 wait complete. 2024-03-16T08:22:30.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.476+0000 7fae3d13b640 1 Processor -- start 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.476+0000 7fae3d13b640 1 -- start start 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae38102a50 0x7fae381980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 0x7fae3819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fae38111bf0 con 0x7fae38103370 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae38111a70 con 0x7fae38102a50 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae38111d70 con 0x7fae3810be20 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35868/0 (socket says 172.21.15.49:35868) 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 0x7fae3819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 -- 172.21.15.49:0/3070026570 learned_addr learned my addr 172.21.15.49:0/3070026570 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae36d9d640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae38102a50 0x7fae381980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 -- 172.21.15.49:0/3070026570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 msgr2=0x7fae3819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 0x7fae3819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 -- 172.21.15.49:0/3070026570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae38102a50 msgr2=0x7fae381980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae38102a50 0x7fae381980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 -- 172.21.15.49:0/3070026570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3819d090 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3759e640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 0x7fae3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae2e59c640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ea60 tx=0x7fae2000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae34d99640 1 -- 172.21.15.49:0/3070026570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000cde0 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3819d380 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae34d99640 1 -- 172.21.15.49:0/3070026570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae200035f0 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae34d99640 1 -- 172.21.15.49:0/3070026570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20010670 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.477+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae381a4c10 con 0x7fae38103370 2024-03-16T08:22:30.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.479+0000 7fae34d99640 1 -- 172.21.15.49:0/3070026570 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fae200037a0 con 0x7fae38103370 2024-03-16T08:22:30.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 Processor -- start 2024-03-16T08:22:30.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 -- start start 2024-03-16T08:22:30.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2648d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26498a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e263f1500 con 0x560e2648da70 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560e26399be0 con 0x560e26496650 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3fe08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560e263ed5a0 con 0x560e2648d150 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae34d99640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 0x7fae240a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae36d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae3759e640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 0x7fae240a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae36d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35870/0 (socket says 172.21.15.49:35870) 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae2e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26498a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae36d9d640 1 -- 172.21.15.49:0/1739550410 learned_addr learned my addr 172.21.15.49:0/1739550410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.480+0000 7fae34d99640 1 -- 172.21.15.49:0/3070026570 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fae200cf100 con 0x7fae38103370 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae3759e640 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2648d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 -- 172.21.15.49:0/1739550410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 msgr2=0x560e2648d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2648d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 -- 172.21.15.49:0/1739550410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 msgr2=0x560e26498a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26498a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 -- 172.21.15.49:0/1739550410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e263ad4b0 con 0x560e2648da70 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae2e59c640 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26498a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae3759e640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 0x7fae240a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae30005fd0 tx=0x7fae3000e5e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fae28005e00 tx=0x7fae28009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12167541b87eff49 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae2d59a640 1 -- 172.21.15.49:0/1739550410 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28043070 con 0x560e2648da70 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae2d59a640 1 -- 172.21.15.49:0/1739550410 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2802ce50 con 0x560e2648da70 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.481+0000 7fae2d59a640 1 -- 172.21.15.49:0/1739550410 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28039040 con 0x560e2648da70 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- 172.21.15.49:0/1739550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 msgr2=0x560e26496110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fae28005e00 tx=0x7fae28009510 comp rx=0 tx=0).stop 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- 172.21.15.49:0/1739550410 shutdown_connections 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26498a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e26496110 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1739550410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2648d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- 172.21.15.49:0/1739550410 >> 172.21.15.49:0/1739550410 conn(0x560e26487880 msgr2=0x560e26487c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- 172.21.15.49:0/1739550410 shutdown_connections 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- 172.21.15.49:0/1739550410 wait complete. 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 -- start start 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.482+0000 7fae3fe08740 1 Processor -- start 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e26493690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648da70 0x560e26493bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e264993e0 con 0x560e26496650 2024-03-16T08:22:30.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560e263ad160 con 0x560e2648da70 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560e26499560 con 0x560e2648d150 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3fe08740 10 client.0 _ll_register_callbacks cb 0x560e26391150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35880/0 (socket says 172.21.15.49:35880) 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 -- 172.21.15.49:0/3149282469 learned_addr learned my addr 172.21.15.49:0/3149282469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae2e59c640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648da70 0x560e26493bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae36d9d640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e26493690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 -- 172.21.15.49:0/3149282469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 msgr2=0x560e26493690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e26493690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 -- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648da70 msgr2=0x560e26493bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648da70 0x560e26493bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 -- 172.21.15.49:0/3149282469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e261cf7d0 con 0x560e26496650 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae3759e640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae30004270 tx=0x7fae300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e32388ded766a44 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:30.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.483+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3001d9d0 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2802ce50 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fae3001db70 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae3001dd70 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 10 client.16917 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e261053f0 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2802ce50 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fae30026070 con 0x560e26496650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 10 client.16917 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 20 client.16917 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e264a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 4 client.16917 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e263fea40 con 0x560e264a6650 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.484+0000 7fae36d9d640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e264a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.485+0000 7fae1e7fc640 10 client.16917 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.485+0000 7fae36d9d640 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e264a8ad0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aeb1e8df89bd916 server_cookie=e6e210906bd780e in_seq=0 out_seq=0 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae1e7fc640 1 -- 172.21.15.49:0/3149282469 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e263fea40 con 0x560e264a6650 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae1e7fc640 10 client.16917 handle_command_reply: tid=0 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 client.16917 shutdown 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 msgr2=0x560e26494110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae30004270 tx=0x7fae300095c0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 shutdown_connections 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e264a8ad0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae1e7fc640 0 client.16917 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26494110 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648da70 0x560e26493bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 --2- 172.21.15.49:0/3149282469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e26493690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 >> 172.21.15.49:0/3149282469 conn(0x560e26487880 msgr2=0x560e264895d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 shutdown_connections 2024-03-16T08:22:30.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 1 -- 172.21.15.49:0/3149282469 wait complete. 2024-03-16T08:22:30.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.486+0000 7fae3fe08740 20 client.16917 trim_cache size 0 max 16384 2024-03-16T08:22:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.505+0000 7fae3fe08740 1 Processor -- start 2024-03-16T08:22:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.505+0000 7fae3fe08740 1 -- start start 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648d150 0x560e26495dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648da70 0x560e264a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fae381afce0 con 0x560e26496650 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ad5b0 con 0x560e2648d150 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3fe08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ab3e0 con 0x560e2648da70 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35884/0 (socket says 172.21.15.49:35884) 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 -- 172.21.15.49:0/2492593081 learned_addr learned my addr 172.21.15.49:0/2492593081 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae2e59c640 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648da70 0x560e264a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae36d9d640 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648d150 0x560e26495dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 -- 172.21.15.49:0/2492593081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648da70 msgr2=0x560e264a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648da70 0x560e264a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 -- 172.21.15.49:0/2492593081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648d150 msgr2=0x560e26495dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648d150 0x560e26495dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 -- 172.21.15.49:0/2492593081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e260c50f0 con 0x560e26496650 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae3759e640 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fae3000d860 tx=0x7fae30000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366d83a9c60b58d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae2cd99640 1 -- 172.21.15.49:0/2492593081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3001da20 con 0x560e26496650 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae2cd99640 1 -- 172.21.15.49:0/2492593081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3001dbc0 con 0x560e26496650 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.506+0000 7fae2cd99640 1 -- 172.21.15.49:0/2492593081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30027810 con 0x560e26496650 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 -- 172.21.15.49:0/2492593081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 msgr2=0x560e26486780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fae3000d860 tx=0x7fae30000f40 comp rx=0 tx=0).stop 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 -- 172.21.15.49:0/2492593081 shutdown_connections 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e26496650 0x560e26486780 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648da70 0x560e264a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 --2- 172.21.15.49:0/2492593081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e2648d150 0x560e26495dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 -- 172.21.15.49:0/2492593081 >> 172.21.15.49:0/2492593081 conn(0x560e26487880 msgr2=0x560e26486070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 -- 172.21.15.49:0/2492593081 shutdown_connections 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.507+0000 7fae3fe08740 1 -- 172.21.15.49:0/2492593081 wait complete. 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 -- start start 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 Processor -- start 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2649ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26490460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fae381aba70 con 0x560e2648da70 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae381acee0 con 0x560e26496650 2024-03-16T08:22:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ac140 con 0x560e2648d150 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3fe08740 10 client.0 _ll_register_callbacks cb 0x7fae3000e7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35896/0 (socket says 172.21.15.49:35896) 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 -- 172.21.15.49:0/1055791108 learned_addr learned my addr 172.21.15.49:0/1055791108 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae3759e640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26490460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2649ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 -- 172.21.15.49:0/1055791108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 msgr2=0x560e2649ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2649ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 -- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 msgr2=0x560e26490460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26490460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 -- 172.21.15.49:0/1055791108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e263b89d0 con 0x560e2648da70 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae2e59c640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fae2001f000 tx=0x7fae20003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afaa7d1505576db server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.508+0000 7fae1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:22:30.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2009f900 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fae200662b0 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3001dbc0 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae200a3f50 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 10 client.16923 fetch_fsmap learned FSMap version 24 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e2650c7f0 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3001dbc0 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fae200a3ac0 con 0x560e2648da70 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 10 client.16923 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 20 client.16923 populate_metadata read hostname 'smithi049' 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e2650ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 4 client.16923 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e263f8d90 con 0x560e264a6650 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.509+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e2650ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae1e7fc640 10 client.16923 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae36d9d640 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e2650ef60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a01cbfb701ad29 server_cookie=323e30c2d8ee9834 in_seq=0 out_seq=0 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae1e7fc640 1 -- 172.21.15.49:0/1055791108 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e263f8d90 con 0x560e264a6650 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae1e7fc640 10 client.16923 handle_command_reply: tid=0 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 client.16923 shutdown 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 msgr2=0x560e2648ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fae2001f000 tx=0x7fae20003f30 comp rx=0 tx=0).stop 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 shutdown_connections 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x560e264a6650 0x560e2650ef60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e26496650 0x560e26490460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e2648da70 0x560e2648ff20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae1e7fc640 0 client.16923 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 --2- 172.21.15.49:0/1055791108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560e2648d150 0x560e2649ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 >> 172.21.15.49:0/1055791108 conn(0x560e26487880 msgr2=0x560e26491b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 shutdown_connections 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 1 -- 172.21.15.49:0/1055791108 wait complete. 2024-03-16T08:22:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.510+0000 7fae3fe08740 20 client.16923 trim_cache size 0 max 16384 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "72628079-597c-4df2-b9b2-05955656c8d5": { 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "72628079-597c-4df2-b9b2-05955656c8d5", 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:22:30.518 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:22:30.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 msgr2=0x7fae240a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 0x7fae240a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae30005fd0 tx=0x7fae3000e5e0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 msgr2=0x7fae38198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:22:30.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ea60 tx=0x7fae2000ef20 comp rx=0 tx=0).stop 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 shutdown_connections 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fae240a2970 0x7fae240a4e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae3810be20 0x7fae3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae38103370 0x7fae38198630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 --2- 172.21.15.49:0/3070026570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae38102a50 0x7fae381980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 >> 172.21.15.49:0/3070026570 conn(0x7fae38077840 msgr2=0x7fae380fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 shutdown_connections 2024-03-16T08:22:30.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:22:30.511+0000 7fae3d13b640 1 -- 172.21.15.49:0/3070026570 wait complete. 2024-03-16T08:22:30.523 INFO:tasks.cephfs.filesystem:scrub status for tag:72628079-597c-4df2-b9b2-05955656c8d5 - {'path': '/', 'tag': '72628079-597c-4df2-b9b2-05955656c8d5', 'options': 'recursive,force'} 2024-03-16T08:22:30.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:30 smithi049 ceph-mon[31942]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:30 smithi129 ceph-mon[36561]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:30 smithi154 ceph-mon[36471]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:31.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:31 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:22:31.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:31 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:31.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:31 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:22:31.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:31 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:31.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:31 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:22:31.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:31 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:31.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:31 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:22:31.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:31 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:31.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:31 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:22:31.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:31 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:31.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:31 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:22:31.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:31 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:22:32.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:32 smithi049 ceph-mon[31942]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:32.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:32 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:22:32.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:32 smithi129 ceph-mon[36561]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:32.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:32 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:22:32.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:32 smithi154 ceph-mon[36471]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:32.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:32 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:22:34.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:34 smithi049 ceph-mon[31942]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:34.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:34 smithi129 ceph-mon[36561]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:34.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:34 smithi154 ceph-mon[36471]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:36.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:36 smithi049 ceph-mon[31942]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:36.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:36 smithi129 ceph-mon[36561]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:36.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:36 smithi154 ceph-mon[36471]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:38.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:38 smithi049 ceph-mon[31942]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:38.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:38 smithi129 ceph-mon[36561]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:38.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:38 smithi154 ceph-mon[36471]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:40.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:40 smithi049 ceph-mon[31942]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:40.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:40 smithi129 ceph-mon[36561]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:40.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:40 smithi154 ceph-mon[36471]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:42.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:42 smithi049 ceph-mon[31942]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:42.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:42 smithi129 ceph-mon[36561]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:42.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:42 smithi154 ceph-mon[36471]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:44.778 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:44 smithi154 ceph-mon[36471]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:44.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:44 smithi049 ceph-mon[31942]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:44.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:44 smithi129 ceph-mon[36561]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:22:46.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:46 smithi049 ceph-mon[31942]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:46.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:46 smithi129 ceph-mon[36561]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:46.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:46 smithi154 ceph-mon[36471]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:22:48.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:48 smithi049 ceph-mon[31942]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:48.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:48 smithi129 ceph-mon[36561]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:48.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:48 smithi154 ceph-mon[36471]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:50.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:50 smithi049 ceph-mon[31942]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:50 smithi129 ceph-mon[36561]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:50.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:50 smithi154 ceph-mon[36471]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:52.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:52 smithi049 ceph-mon[31942]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:52 smithi129 ceph-mon[36561]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:52.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:52 smithi154 ceph-mon[36471]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:54.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:54 smithi049 ceph-mon[31942]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:54.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:54 smithi129 ceph-mon[36561]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:54.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:54 smithi154 ceph-mon[36471]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:56.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:56 smithi049 ceph-mon[31942]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:56 smithi129 ceph-mon[36561]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:56.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:56 smithi154 ceph-mon[36471]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:22:58.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:22:58 smithi049 ceph-mon[31942]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:58.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:22:58 smithi129 ceph-mon[36561]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:22:58.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:22:58 smithi154 ceph-mon[36471]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:00.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:23:00.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:23:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 -- 172.21.15.49:0/2656729809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700101750 msgr2=0x7f9700101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 --2- 172.21.15.49:0/2656729809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700101750 0x7f9700101bd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009930 tx=0x7f96f402bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 -- 172.21.15.49:0/2656729809 shutdown_connections 2024-03-16T08:23:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 --2- 172.21.15.49:0/2656729809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700102110 0x7f970010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 --2- 172.21.15.49:0/2656729809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700101750 0x7f9700101bd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 --2- 172.21.15.49:0/2656729809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700107760 0x7f9700107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.656+0000 7f9704df4640 1 -- 172.21.15.49:0/2656729809 >> 172.21.15.49:0/2656729809 conn(0x7f97000fd240 msgr2=0x7f97000ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.657+0000 7f9704df4640 1 -- 172.21.15.49:0/2656729809 shutdown_connections 2024-03-16T08:23:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.657+0000 7f9704df4640 1 -- 172.21.15.49:0/2656729809 wait complete. 2024-03-16T08:23:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.657+0000 7f9704df4640 1 Processor -- start 2024-03-16T08:23:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.657+0000 7f9704df4640 1 -- start start 2024-03-16T08:23:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700101750 0x7f9700198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700107760 0x7f970019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9700111b70 con 0x7f9700102110 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97001119f0 con 0x7f9700101750 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9700111cf0 con 0x7f9700107760 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36662/0 (socket says 172.21.15.49:36662) 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 -- 172.21.15.49:0/629617036 learned_addr learned my addr 172.21.15.49:0/629617036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fed9d640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700107760 0x7f970019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fe59c640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700101750 0x7f9700198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 -- 172.21.15.49:0/629617036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700107760 msgr2=0x7f970019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700107760 0x7f970019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 -- 172.21.15.49:0/629617036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700101750 msgr2=0x7f9700198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700101750 0x7f9700198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 -- 172.21.15.49:0/629617036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970019d100 con 0x7f9700102110 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009230 tx=0x7f96f402cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96ef7fe640 1 -- 172.21.15.49:0/629617036 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4041070 con 0x7f9700102110 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970019d390 con 0x7f9700102110 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96ef7fe640 1 -- 172.21.15.49:0/629617036 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4004450 con 0x7f9700102110 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.658+0000 7f96ef7fe640 1 -- 172.21.15.49:0/629617036 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4034440 con 0x7f9700102110 2024-03-16T08:23:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.659+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a4c80 con 0x7f9700102110 2024-03-16T08:23:00.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f96ef7fe640 1 -- 172.21.15.49:0/629617036 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f96f4044070 con 0x7f9700102110 2024-03-16T08:23:00.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 Processor -- start 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 -- start start 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 0x558d7725e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d7726de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d771c2500 con 0x558d7726ba60 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d7716abe0 con 0x558d7725de00 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.661+0000 7f97079dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558d771be5a0 con 0x558d7725e720 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96ef7fe640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 0x7f96e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d7726de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:45106/0 (socket says 172.21.15.49:45106) 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 -- 172.21.15.49:0/3811506278 learned_addr learned my addr 172.21.15.49:0/3811506278 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d7726de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36670/0 (socket says 172.21.15.49:36670) 2024-03-16T08:23:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fed9d640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 0x7f96e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fed9d640 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 0x558d7725e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 -- 172.21.15.49:0/3811506278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 msgr2=0x558d7725e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 0x558d7725e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96ef7fe640 1 -- 172.21.15.49:0/629617036 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f96f40f06e0 con 0x7f9700102110 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 -- 172.21.15.49:0/3811506278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 msgr2=0x558d7726de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d7726de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 -- 172.21.15.49:0/3811506278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d7717e4b0 con 0x558d7725e720 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fed9d640 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 0x558d7725e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009c70 tx=0x7f96f0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbbde72ef23da81 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96fed9d640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 0x7f96e80a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f8009b60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96ecff9640 1 -- 172.21.15.49:0/3811506278 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0004230 con 0x558d7725e720 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96ecff9640 1 -- 172.21.15.49:0/3811506278 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f00043d0 con 0x558d7725e720 2024-03-16T08:23:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.662+0000 7f96ecff9640 1 -- 172.21.15.49:0/3811506278 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0019740 con 0x558d7725e720 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 -- 172.21.15.49:0/3811506278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 msgr2=0x558d7726b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009c70 tx=0x7f96f0011810 comp rx=0 tx=0).stop 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 -- 172.21.15.49:0/3811506278 shutdown_connections 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d7726de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d7726b520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 --2- 172.21.15.49:0/3811506278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725de00 0x558d7725e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 -- 172.21.15.49:0/3811506278 >> 172.21.15.49:0/3811506278 conn(0x558d77258880 msgr2=0x558d77258c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 -- 172.21.15.49:0/3811506278 shutdown_connections 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.663+0000 7f97079dd740 1 -- 172.21.15.49:0/3811506278 wait complete. 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 -- start start 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 Processor -- start 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d77268a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d772694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558d7726e8a0 con 0x558d7725e720 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d7717e160 con 0x558d7726ba60 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558d7726ea20 con 0x558d7725de00 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f97079dd740 10 client.0 _ll_register_callbacks cb 0x558d77162150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36678/0 (socket says 172.21.15.49:36678) 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d77268a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 -- 172.21.15.49:0/3705206529 learned_addr learned my addr 172.21.15.49:0/3705206529 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fed9d640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d772694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 -- 172.21.15.49:0/3705206529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 msgr2=0x558d77268a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d77268a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 -- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 msgr2=0x558d772694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d772694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fdd9b640 1 -- 172.21.15.49:0/3705206529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d76fa07d0 con 0x558d7725e720 2024-03-16T08:23:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.664+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d77268a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009430 tx=0x7f96f40d1130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d35144dfaddafc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f404b040 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f96f40ee100 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f00043d0 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f96f40bca90 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 10 client.16935 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558d76ed63f0 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f00043d0 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f96f40c51b0 con 0x558d7725e720 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 10 client.16935 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 20 client.16935 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d7727ba30 0x558d7727de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 4 client.16935 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.665+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d771cfa40 con 0x558d7727ba30 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.666+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d7727ba30 0x558d7727de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.666+0000 7f96fe59c640 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d7727ba30 0x558d7727de80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d646e61c38522aa server_cookie=e18dc372be01d018 in_seq=0 out_seq=0 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.666+0000 7f96e5ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f96e5ffb640 1 -- 172.21.15.49:0/3705206529 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558d771cfa40 con 0x558d7727ba30 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f96e5ffb640 10 client.16935 handle_command_reply: tid=0 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 client.16935 shutdown 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 msgr2=0x558d77268f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009430 tx=0x7f96f40d1130 comp rx=0 tx=0).stop 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 shutdown_connections 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d7727ba30 0x558d7727de80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d772694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f96e5ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725e720 0x558d77268f70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 --2- 172.21.15.49:0/3705206529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d77268a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 >> 172.21.15.49:0/3705206529 conn(0x558d77258880 msgr2=0x558d7725a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 shutdown_connections 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 1 -- 172.21.15.49:0/3705206529 wait complete. 2024-03-16T08:23:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.667+0000 7f97079dd740 20 client.16935 trim_cache size 0 max 16384 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 Processor -- start 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 -- start start 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d7726b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 0x558d772776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97001adad0 con 0x558d7726ba60 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97001ab3a0 con 0x558d7725e720 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f97079dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97001a91d0 con 0x558d7725de00 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36686/0 (socket says 172.21.15.49:36686) 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 -- 172.21.15.49:0/1884137222 learned_addr learned my addr 172.21.15.49:0/1884137222 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fe59c640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d7726b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 0x558d772776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 -- 172.21.15.49:0/1884137222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 msgr2=0x558d7726b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d7726b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 -- 172.21.15.49:0/1884137222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 msgr2=0x558d772776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 0x558d772776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 -- 172.21.15.49:0/1884137222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d76e960f0 con 0x558d7726ba60 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fe59c640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d7726b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:00.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 0x558d772776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.685+0000 7f96fed9d640 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f96f8012660 tx=0x7f96f8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f75cc38df90a296 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f96e7fff640 1 -- 172.21.15.49:0/1884137222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f802e070 con 0x558d7726ba60 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f96e7fff640 1 -- 172.21.15.49:0/1884137222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f801fe50 con 0x558d7726ba60 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f96e7fff640 1 -- 172.21.15.49:0/1884137222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8029c40 con 0x558d7726ba60 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 -- 172.21.15.49:0/1884137222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 msgr2=0x558d77283790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f96f8012660 tx=0x7f96f8014040 comp rx=0 tx=0).stop 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 -- 172.21.15.49:0/1884137222 shutdown_connections 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7726ba60 0x558d77283790 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7725e720 0x558d772776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 --2- 172.21.15.49:0/1884137222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725de00 0x558d7726b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 -- 172.21.15.49:0/1884137222 >> 172.21.15.49:0/1884137222 conn(0x558d77258880 msgr2=0x558d7725a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 -- 172.21.15.49:0/1884137222 shutdown_connections 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.686+0000 7f97079dd740 1 -- 172.21.15.49:0/1884137222 wait complete. 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 -- start start 2024-03-16T08:23:00.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 Processor -- start 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d77273dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d7726fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97001a9860 con 0x558d7725de00 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97001aacd0 con 0x558d7726ba60 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97001a9f30 con 0x558d7725e720 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f97079dd740 10 client.0 _ll_register_callbacks cb 0x7f96f4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36688/0 (socket says 172.21.15.49:36688) 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 -- 172.21.15.49:0/2384759851 learned_addr learned my addr 172.21.15.49:0/2384759851 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d77273dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fed9d640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d7726fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 -- 172.21.15.49:0/2384759851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 msgr2=0x558d77273dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d77273dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 -- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 msgr2=0x558d7726fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d7726fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 -- 172.21.15.49:0/2384759851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d771899d0 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96fe59c640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f96f0003ee0 tx=0x7f96f0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be15dc6e237560 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.687+0000 7f96e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00097a0 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f96f00035f0 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f801fe50 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f96f00037f0 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 10 client.16941 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558d772e2620 con 0x558d7725de00 2024-03-16T08:23:00.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f801fe50 con 0x558d7725de00 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f96f0018070 con 0x558d7725de00 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 10 client.16941 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 20 client.16941 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d772e2ba0 0x558d772e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 4 client.16941 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d771c9d90 con 0x558d772e2ba0 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.688+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d772e2ba0 0x558d772e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f96e5ffb640 10 client.16941 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f96fdd9b640 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d772e2ba0 0x558d772e4f90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc33f0197ae448b4 server_cookie=e2daf6641f641fbd in_seq=0 out_seq=0 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f96e5ffb640 1 -- 172.21.15.49:0/2384759851 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d771c9d90 con 0x558d772e2ba0 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f96e5ffb640 10 client.16941 handle_command_reply: tid=0 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 client.16941 shutdown 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 msgr2=0x558d77273880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f96f0003ee0 tx=0x7f96f0005d30 comp rx=0 tx=0).stop 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 shutdown_connections 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558d772e2ba0 0x558d772e4f90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d7726ba60 0x558d7726fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558d7725e720 0x558d77273dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 --2- 172.21.15.49:0/2384759851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d7725de00 0x558d77273880 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 >> 172.21.15.49:0/2384759851 conn(0x558d77258880 msgr2=0x558d77257070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f96e5ffb640 0 client.16941 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 shutdown_connections 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 1 -- 172.21.15.49:0/2384759851 wait complete. 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.689+0000 7f97079dd740 20 client.16941 trim_cache size 0 max 16384 2024-03-16T08:23:00.697 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:00.698 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:23:00.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:23:00.698 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:00.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 msgr2=0x7f96e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 0x7f96e80a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f8009b60 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 msgr2=0x7f9700198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009230 tx=0x7f96f402cba0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 shutdown_connections 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f96e80a2970 0x7f96e80a4e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700107760 0x7f970019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9700102110 0x7f9700198650 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 --2- 172.21.15.49:0/629617036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9700101750 0x7f9700198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 >> 172.21.15.49:0/629617036 conn(0x7f97000fd240 msgr2=0x7f97000fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 shutdown_connections 2024-03-16T08:23:00.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.690+0000 7f9704df4640 1 -- 172.21.15.49:0/629617036 wait complete. 2024-03-16T08:23:00.701 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:23:00.701 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:23:00.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:00 smithi049 ceph-mon[31942]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.809+0000 7f2b8ce97640 1 -- 172.21.15.49:0/3742733654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88102430 msgr2=0x7f2b8810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.809+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/3742733654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88102430 0x7f2b8810e690 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009ca0 tx=0x7f2b8002c180 comp rx=0 tx=0).stop 2024-03-16T08:23:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 -- 172.21.15.49:0/3742733654 shutdown_connections 2024-03-16T08:23:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/3742733654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88102430 0x7f2b8810e690 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/3742733654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88101a70 0x7f2b88101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/3742733654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88107980 0x7f2b88107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 -- 172.21.15.49:0/3742733654 >> 172.21.15.49:0/3742733654 conn(0x7f2b880797d0 msgr2=0x7f2b88079be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:00.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 -- 172.21.15.49:0/3742733654 shutdown_connections 2024-03-16T08:23:00.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 -- 172.21.15.49:0/3742733654 wait complete. 2024-03-16T08:23:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 Processor -- start 2024-03-16T08:23:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.810+0000 7f2b8ce97640 1 -- start start 2024-03-16T08:23:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88102430 0x7f2b88074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88107980 0x7f2b88074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88111c00 con 0x7f2b88101a70 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88111a80 con 0x7f2b88102430 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88111d80 con 0x7f2b88107980 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36712/0 (socket says 172.21.15.49:36712) 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 -- 172.21.15.49:0/1070397608 learned_addr learned my addr 172.21.15.49:0/1070397608 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b86d9d640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88107980 0x7f2b88074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b85d9b640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88102430 0x7f2b88074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 -- 172.21.15.49:0/1070397608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88107980 msgr2=0x7f2b88074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88107980 0x7f2b88074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 -- 172.21.15.49:0/1070397608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88102430 msgr2=0x7f2b88074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88102430 0x7f2b88074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 -- 172.21.15.49:0/1070397608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b881a65e0 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8659c640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800ea60 tx=0x7f2b7800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800cde0 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b780035f0 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881a68d0 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.811+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78010670 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.812+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881a6e20 con 0x7f2b88101a70 2024-03-16T08:23:00.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.812+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b880685d0 con 0x7f2b88101a70 2024-03-16T08:23:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.813+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2b780037a0 con 0x7f2b88101a70 2024-03-16T08:23:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.814+0000 7f2b777fe640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 0x7f2b700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.814+0000 7f2b85d9b640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 0x7f2b700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.814+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f2b780cf0b0 con 0x7f2b88101a70 2024-03-16T08:23:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.815+0000 7f2b85d9b640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 0x7f2b700a4e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b88075570 tx=0x7f2b7c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:00.819+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b7809be60 con 0x7f2b88101a70 2024-03-16T08:23:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:00 smithi129 ceph-mon[36561]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:00.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:00 smithi154 ceph-mon[36471]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.041+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b88109ce0 con 0x7f2b88101a70 2024-03-16T08:23:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.041+0000 7f2b777fe640 1 -- 172.21.15.49:0/1070397608 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f2b7809b760 con 0x7f2b88101a70 2024-03-16T08:23:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:23:01.042 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:23:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 msgr2=0x7f2b700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 0x7f2b700a4e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b88075570 tx=0x7f2b7c005f90 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 msgr2=0x7f2b88073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800ea60 tx=0x7f2b7800ef20 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 shutdown_connections 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b700a2970 0x7f2b700a4e30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b88107980 0x7f2b88074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b88102430 0x7f2b88074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 --2- 172.21.15.49:0/1070397608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b88101a70 0x7f2b88073ae0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 >> 172.21.15.49:0/1070397608 conn(0x7f2b880797d0 msgr2=0x7f2b881006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 shutdown_connections 2024-03-16T08:23:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.043+0000 7f2b8ce97640 1 -- 172.21.15.49:0/1070397608 wait complete. 2024-03-16T08:23:01.055 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 -- 172.21.15.49:0/1549511486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 msgr2=0x7faf5010e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 --2- 172.21.15.49:0/1549511486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf5010e290 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7faf48009ca0 tx=0x7faf4802c180 comp rx=0 tx=0).stop 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 -- 172.21.15.49:0/1549511486 shutdown_connections 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 --2- 172.21.15.49:0/1549511486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf5010e290 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 --2- 172.21.15.49:0/1549511486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf5010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 --2- 172.21.15.49:0/1549511486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 0x7faf50102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.165+0000 7faf56da9640 1 -- 172.21.15.49:0/1549511486 >> 172.21.15.49:0/1549511486 conn(0x7faf50077840 msgr2=0x7faf50077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- 172.21.15.49:0/1549511486 shutdown_connections 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- 172.21.15.49:0/1549511486 wait complete. 2024-03-16T08:23:01.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 Processor -- start 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- start start 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 0x7faf501006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf50111bf0 con 0x7faf5010be20 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf50111a70 con 0x7faf50103370 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf56da9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf50111d70 con 0x7faf50102a50 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42844/0 (socket says 172.21.15.49:42844) 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf4ffff640 1 -- 172.21.15.49:0/766906684 learned_addr learned my addr 172.21.15.49:0/766906684 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36740/0 (socket says 172.21.15.49:36740) 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.166+0000 7faf54b45640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 0x7faf501006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 -- 172.21.15.49:0/766906684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 msgr2=0x7faf501006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 0x7faf501006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 -- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 msgr2=0x7faf50101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 -- 172.21.15.49:0/766906684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf501a6590 con 0x7faf50103370 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf55346640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4ffff640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4dffb640 1 -- 172.21.15.49:0/766906684 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400cd80 con 0x7faf50103370 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4dffb640 1 -- 172.21.15.49:0/766906684 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf44010910 con 0x7faf50103370 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf4dffb640 1 -- 172.21.15.49:0/766906684 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44018990 con 0x7faf50103370 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf501a6880 con 0x7faf50103370 2024-03-16T08:23:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.167+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf501a6dd0 con 0x7faf50103370 2024-03-16T08:23:01.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.169+0000 7faf4dffb640 1 -- 172.21.15.49:0/766906684 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7faf44010ac0 con 0x7faf50103370 2024-03-16T08:23:01.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 Processor -- start 2024-03-16T08:23:01.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 -- start start 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997d9bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 0x561997da47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997da4d30 0x561997da7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561997cff490 con 0x561997d9b830 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561997ca7c90 con 0x561997da4d30 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf59990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561997cfb530 con 0x561997d9c150 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.170+0000 7faf4dffb640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 0x7faf380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 0x561997da47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997d9bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997d9bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36746/0 (socket says 172.21.15.49:36746) 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf55346640 1 -- 172.21.15.49:0/1632079834 learned_addr learned my addr 172.21.15.49:0/1632079834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf4ffff640 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997da4d30 0x561997da7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf55346640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 0x7faf380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf4dffb640 1 -- 172.21.15.49:0/766906684 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7faf440cf120 con 0x7faf50103370 2024-03-16T08:23:01.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 -- 172.21.15.49:0/1632079834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997da4d30 msgr2=0x561997da7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997da4d30 0x561997da7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 -- 172.21.15.49:0/1632079834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 msgr2=0x561997d9bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997d9bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 -- 172.21.15.49:0/1632079834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561997d62500 con 0x561997d9c150 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf55346640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 0x7faf380a4e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf48037040 tx=0x7faf48005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf54b45640 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 0x561997da47f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf40009c20 tx=0x7faf40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40efe0db05685983 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf36ffd640 1 -- 172.21.15.49:0/1632079834 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4001e070 con 0x561997d9c150 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf36ffd640 1 -- 172.21.15.49:0/1632079834 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf40009dd0 con 0x561997d9c150 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.171+0000 7faf36ffd640 1 -- 172.21.15.49:0/1632079834 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400187d0 con 0x561997d9c150 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- 172.21.15.49:0/1632079834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 msgr2=0x561997da47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 0x561997da47f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf40009c20 tx=0x7faf40011810 comp rx=0 tx=0).stop 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- 172.21.15.49:0/1632079834 shutdown_connections 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997da4d30 0x561997da7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9c150 0x561997da47f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 --2- 172.21.15.49:0/1632079834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997d9bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- 172.21.15.49:0/1632079834 >> 172.21.15.49:0/1632079834 conn(0x561997d95f60 msgr2=0x561997d96340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- 172.21.15.49:0/1632079834 shutdown_connections 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- 172.21.15.49:0/1632079834 wait complete. 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 -- start start 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.172+0000 7faf59990740 1 Processor -- start 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997da2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997da2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561997da7b20 con 0x561997da4d30 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561997cbb230 con 0x561997d9c150 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561997da7ca0 con 0x561997d9b830 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf59990740 10 client.0 _ll_register_callbacks cb 0x561997cf7010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997da2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997da2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:45188/0 (socket says 172.21.15.49:45188) 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 -- 172.21.15.49:0/1291467891 learned_addr learned my addr 172.21.15.49:0/1291467891 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 -- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 msgr2=0x561997da2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997da2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 -- 172.21.15.49:0/1291467891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 msgr2=0x561997da2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997da2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf54b45640 1 -- 172.21.15.49:0/1291467891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561997cca100 con 0x561997d9b830 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.173+0000 7faf55346640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997da2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf54b45640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf400183f0 tx=0x7faf40005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4049d212dbd77963 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4001e040 con 0x561997d9b830 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7faf40005050 con 0x561997d9b830 2024-03-16T08:23:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf40009dd0 con 0x561997d9b830 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf40005250 con 0x561997d9b830 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 10 client.26842 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561997a133f0 con 0x561997d9b830 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf40009dd0 con 0x561997d9b830 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7faf40019510 con 0x561997d9b830 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 10 client.26842 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 10 client.26842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 20 client.26842 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997db73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 4 client.26842 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.174+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561997d0c9d0 con 0x561997db5000 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.175+0000 7faf4ffff640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997db73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.175+0000 7faf4ffff640 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997db73f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97065afd68686c3c server_cookie=c127efe6c551a2bf in_seq=0 out_seq=0 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.175+0000 7faf2ffff640 10 client.26842 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf2ffff640 1 -- 172.21.15.49:0/1291467891 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7faf440aa9c0 con 0x561997db5000 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf2ffff640 10 client.26842 handle_command_reply: tid=0 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 client.26842 shutdown 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 msgr2=0x561997da1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf400183f0 tx=0x7faf40005db0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 shutdown_connections 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997db73f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997da2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf2ffff640 0 client.26842 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997da2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 --2- 172.21.15.49:0/1291467891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997da1dc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 >> 172.21.15.49:0/1291467891 conn(0x561997d95f60 msgr2=0x561997d97cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 shutdown_connections 2024-03-16T08:23:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 1 -- 172.21.15.49:0/1291467891 wait complete. 2024-03-16T08:23:01.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.176+0000 7faf59990740 20 client.26842 trim_cache size 0 max 16384 2024-03-16T08:23:01.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 Processor -- start 2024-03-16T08:23:01.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 -- start start 2024-03-16T08:23:01.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997db08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997da4d30 0x561997dbcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf501b1f00 con 0x561997d9b830 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf501af7d0 con 0x561997d9c150 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf59990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf501ad600 con 0x561997da4d30 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36764/0 (socket says 172.21.15.49:36764) 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 -- 172.21.15.49:0/655241375 learned_addr learned my addr 172.21.15.49:0/655241375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf4ffff640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997db08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf55346640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997da4d30 0x561997dbcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 -- 172.21.15.49:0/655241375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997da4d30 msgr2=0x561997dbcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997da4d30 0x561997dbcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 -- 172.21.15.49:0/655241375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 msgr2=0x561997db08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997db08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 -- 172.21.15.49:0/655241375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561997d91640 con 0x561997d9b830 2024-03-16T08:23:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf4ffff640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997db08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.190+0000 7faf54b45640 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7faf40009b40 tx=0x7faf400035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9c4b12bf98cb7e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf367fc640 1 -- 172.21.15.49:0/655241375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400053f0 con 0x561997d9b830 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf367fc640 1 -- 172.21.15.49:0/655241375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf40005590 con 0x561997d9b830 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf367fc640 1 -- 172.21.15.49:0/655241375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40003bf0 con 0x561997d9b830 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 -- 172.21.15.49:0/655241375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 msgr2=0x561997da44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7faf40009b40 tx=0x7faf400035f0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 -- 172.21.15.49:0/655241375 shutdown_connections 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997da4d30 0x561997dbcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997db08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 --2- 172.21.15.49:0/655241375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997d9b830 0x561997da44f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 -- 172.21.15.49:0/655241375 >> 172.21.15.49:0/655241375 conn(0x561997d95f60 msgr2=0x561997d97300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 -- 172.21.15.49:0/655241375 shutdown_connections 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.191+0000 7faf59990740 1 -- 172.21.15.49:0/655241375 wait complete. 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 -- start start 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 Processor -- start 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997dacef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997d9e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf501adc90 con 0x561997da4d30 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf501af100 con 0x561997d9c150 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf501ae360 con 0x561997d9b830 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf59990740 10 client.0 _ll_register_callbacks cb 0x7faf4001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf55346640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36772/0 (socket says 172.21.15.49:36772) 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf55346640 1 -- 172.21.15.49:0/3257273709 learned_addr learned my addr 172.21.15.49:0/3257273709 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf54b45640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997dacef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.192+0000 7faf4ffff640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997d9e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 -- 172.21.15.49:0/3257273709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 msgr2=0x561997dacef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997dacef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 -- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 msgr2=0x561997d9e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997d9e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 -- 172.21.15.49:0/3257273709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561997add7d0 con 0x561997da4d30 2024-03-16T08:23:01.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf55346640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faf48009ca0 tx=0x7faf4802c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aadceede7225a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48055070 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7faf48003e70 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf40005590 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf48061050 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 10 client.16965 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561997e1abc0 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf40005590 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7faf480517b0 con 0x561997da4d30 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 10 client.16965 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 20 client.16965 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997e1d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 4 client.16965 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.193+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561997d06d20 con 0x561997db5000 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf54b45640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997e1d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf54b45640 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997e1d380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cc1fadd93748a83 server_cookie=3358387dfc698663 in_seq=0 out_seq=0 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf2ffff640 10 client.16965 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf2ffff640 1 -- 172.21.15.49:0/3257273709 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561997d06d20 con 0x561997db5000 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf2ffff640 10 client.16965 handle_command_reply: tid=0 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 client.16965 shutdown 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 msgr2=0x561997d9e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faf48009ca0 tx=0x7faf4802c180 comp rx=0 tx=0).stop 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 shutdown_connections 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x561997db5000 0x561997e1d380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561997da4d30 0x561997d9e750 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf2ffff640 0 client.16965 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561997d9c150 0x561997d9e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 --2- 172.21.15.49:0/3257273709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561997d9b830 0x561997dacef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 >> 172.21.15.49:0/3257273709 conn(0x561997d95f60 msgr2=0x561997da2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 shutdown_connections 2024-03-16T08:23:01.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.194+0000 7faf59990740 1 -- 172.21.15.49:0/3257273709 wait complete. 2024-03-16T08:23:01.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf59990740 20 client.16965 trim_cache size 0 max 16384 2024-03-16T08:23:01.201 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 msgr2=0x7faf380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 0x7faf380a4e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf48037040 tx=0x7faf48005c50 comp rx=0 tx=0).stop 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 msgr2=0x7faf50100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 shutdown_connections 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7faf380a2970 0x7faf380a4e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf5010be20 0x7faf50101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf50103370 0x7faf50100be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 --2- 172.21.15.49:0/766906684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf50102a50 0x7faf501006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 >> 172.21.15.49:0/766906684 conn(0x7faf50077840 msgr2=0x7faf500fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 shutdown_connections 2024-03-16T08:23:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.195+0000 7faf56da9640 1 -- 172.21.15.49:0/766906684 wait complete. 2024-03-16T08:23:01.206 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 -- 172.21.15.49:0/3163622489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0101780 msgr2=0x7fbbc0101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/3163622489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0101780 0x7fbbc0101c00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009930 tx=0x7fbbb402bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 -- 172.21.15.49:0/3163622489 shutdown_connections 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/3163622489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 0x7fbbc010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/3163622489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0101780 0x7fbbc0101c00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/3163622489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0107790 0x7fbbc0107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 -- 172.21.15.49:0/3163622489 >> 172.21.15.49:0/3163622489 conn(0x7fbbc00fd210 msgr2=0x7fbbc00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 -- 172.21.15.49:0/3163622489 shutdown_connections 2024-03-16T08:23:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.316+0000 7fbbc62d1640 1 -- 172.21.15.49:0/3163622489 wait complete. 2024-03-16T08:23:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 Processor -- start 2024-03-16T08:23:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 -- start start 2024-03-16T08:23:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0101780 0x7fbbc01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 0x7fbbc0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0111b60 con 0x7fbbc0107790 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01119e0 con 0x7fbbc0102140 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc62d1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0111ce0 con 0x7fbbc0101780 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc486e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 0x7fbbc0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc486e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36790/0 (socket says 172.21.15.49:36790) 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbc486e640 1 -- 172.21.15.49:0/1654030847 learned_addr learned my addr 172.21.15.49:0/1654030847 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.317+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0101780 0x7fbbc01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 -- 172.21.15.49:0/1654030847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0101780 msgr2=0x7fbbc01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0101780 0x7fbbc01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 -- 172.21.15.49:0/1654030847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 msgr2=0x7fbbc0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 0x7fbbc0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 -- 172.21.15.49:0/1654030847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc019d080 con 0x7fbbc0107790 2024-03-16T08:23:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc486e640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800eb10 tx=0x7fbbb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbbd7fa640 1 -- 172.21.15.49:0/1654030847 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004b20 con 0x7fbbc0107790 2024-03-16T08:23:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbbd7fa640 1 -- 172.21.15.49:0/1654030847 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb8004cc0 con 0x7fbbc0107790 2024-03-16T08:23:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc019d370 con 0x7fbbc0107790 2024-03-16T08:23:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbbd7fa640 1 -- 172.21.15.49:0/1654030847 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8010760 con 0x7fbbc0107790 2024-03-16T08:23:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.318+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01a4c00 con 0x7fbbc0107790 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.320+0000 7fbbbd7fa640 1 -- 172.21.15.49:0/1654030847 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fbbb8005020 con 0x7fbbc0107790 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.320+0000 7fbbc8f99740 1 Processor -- start 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.320+0000 7fbbc8f99740 1 -- start start 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbd7fa640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 0x7fbba80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee29b6d490 con 0x55ee29c09830 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ee29b15c90 con 0x55ee29c12d30 2024-03-16T08:23:01.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee29b69530 con 0x55ee29c0a150 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 0x7fbba80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36802/0 (socket says 172.21.15.49:36802) 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbf7fe640 1 -- 172.21.15.49:0/3159120470 learned_addr learned my addr 172.21.15.49:0/3159120470 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbd7fa640 1 -- 172.21.15.49:0/1654030847 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbbb80cf390 con 0x7fbbc0107790 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 -- 172.21.15.49:0/3159120470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 msgr2=0x55ee29c127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 -- 172.21.15.49:0/3159120470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 msgr2=0x55ee29c09c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 -- 172.21.15.49:0/3159120470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee29bd0500 con 0x55ee29c12d30 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 0x7fbba80a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009b50 tx=0x7fbbb0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbbc486e640 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c15120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80ccb30 tx=0x7fbbb800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0c6c8c832601bb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.321+0000 7fbba67fc640 1 -- 172.21.15.49:0/3159120470 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb809b3a0 con 0x55ee29c12d30 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbba67fc640 1 -- 172.21.15.49:0/3159120470 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb809c140 con 0x55ee29c12d30 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbba67fc640 1 -- 172.21.15.49:0/3159120470 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8061130 con 0x55ee29c12d30 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3159120470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 msgr2=0x55ee29c15120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c15120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80ccb30 tx=0x7fbbb800cf70 comp rx=0 tx=0).stop 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3159120470 shutdown_connections 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c15120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3159120470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c09c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3159120470 >> 172.21.15.49:0/3159120470 conn(0x55ee29c03f60 msgr2=0x55ee29c04340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3159120470 shutdown_connections 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.322+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3159120470 wait complete. 2024-03-16T08:23:01.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 -- start start 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 Processor -- start 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c10300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee29c15b20 con 0x55ee29c09830 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ee29b29230 con 0x55ee29c12d30 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee29c15ca0 con 0x55ee29c0a150 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc8f99740 10 client.0 _ll_register_callbacks cb 0x55ee29b65010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36816/0 (socket says 172.21.15.49:36816) 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbbffff640 1 -- 172.21.15.49:0/1116714990 learned_addr learned my addr 172.21.15.49:0/1116714990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbc486e640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c10300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.323+0000 7fbbbffff640 1 -- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 msgr2=0x55ee29c10300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c10300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbbffff640 1 -- 172.21.15.49:0/1116714990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 msgr2=0x55ee29c10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbbffff640 1 -- 172.21.15.49:0/1116714990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee29b38100 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbbffff640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0003970 tx=0x7fbbb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88df77e3eaf4abd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000ecc0 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbbb001e420 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb809c140 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbbb000ee60 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 10 client.16977 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ee298813f0 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb809c140 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbbb00307a0 con 0x55ee29c09830 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 10 client.16977 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:23:01.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 20 client.16977 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c230c0 0x55ee29c25520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 4 client.16977 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.324+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee29b7a9d0 con 0x55ee29c230c0 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.325+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c230c0 0x55ee29c25520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.325+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c230c0 0x55ee29c25520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaa5bdcb240a79e9 server_cookie=1693255bd2a9050 in_seq=0 out_seq=0 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.325+0000 7fbb9f7fe640 10 client.16977 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/1116714990 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ee29b7a9d0 con 0x55ee29c230c0 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbb9f7fe640 10 client.16977 handle_command_reply: tid=0 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 client.16977 shutdown 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 msgr2=0x55ee29c0fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0003970 tx=0x7fbbb0004730 comp rx=0 tx=0).stop 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 shutdown_connections 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c230c0 0x55ee29c25520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbb9f7fe640 0 client.16977 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c10300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/1116714990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c0fdc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 >> 172.21.15.49:0/1116714990 conn(0x55ee29c03f60 msgr2=0x55ee29c05cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 shutdown_connections 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 1 -- 172.21.15.49:0/1116714990 wait complete. 2024-03-16T08:23:01.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.326+0000 7fbbc8f99740 20 client.16977 trim_cache size 0 max 16384 2024-03-16T08:23:01.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 Processor -- start 2024-03-16T08:23:01.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 -- start start 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01afd70 con 0x55ee29c09830 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01ad640 con 0x55ee29c12d30 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.339+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01ab470 con 0x55ee29c0a150 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36824/0 (socket says 172.21.15.49:36824) 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 -- 172.21.15.49:0/2733927269 learned_addr learned my addr 172.21.15.49:0/2733927269 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbc486e640 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 -- 172.21.15.49:0/2733927269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 msgr2=0x55ee29c1e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c1e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 -- 172.21.15.49:0/2733927269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 msgr2=0x55ee29c02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 -- 172.21.15.49:0/2733927269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2994b7d0 con 0x55ee29c09830 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbbbffff640 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00059f0 tx=0x7fbbb000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d3d305b43769c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbba5ffb640 1 -- 172.21.15.49:0/2733927269 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb003f070 con 0x55ee29c09830 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbba5ffb640 1 -- 172.21.15.49:0/2733927269 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb0030960 con 0x55ee29c09830 2024-03-16T08:23:01.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.340+0000 7fbba5ffb640 1 -- 172.21.15.49:0/2733927269 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0036040 con 0x55ee29c09830 2024-03-16T08:23:01.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- 172.21.15.49:0/2733927269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 msgr2=0x55ee29c124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00059f0 tx=0x7fbbb000e780 comp rx=0 tx=0).stop 2024-03-16T08:23:01.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- 172.21.15.49:0/2733927269 shutdown_connections 2024-03-16T08:23:01.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c12d30 0x55ee29c02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c0a150 0x55ee29c1e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/2733927269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c09830 0x55ee29c124f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- 172.21.15.49:0/2733927269 >> 172.21.15.49:0/2733927269 conn(0x55ee29c03f60 msgr2=0x55ee29c02750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- 172.21.15.49:0/2733927269 shutdown_connections 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- 172.21.15.49:0/2733927269 wait complete. 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 -- start start 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.341+0000 7fbbc8f99740 1 Processor -- start 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 0x55ee29c16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c12d30 0x55ee29c174f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01abb00 con 0x55ee29c0a150 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01acf70 con 0x55ee29c09830 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01ac1d0 con 0x55ee29c12d30 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 10 client.0 _ll_register_callbacks cb 0x7fbbb0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 0x55ee29c16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36836/0 (socket says 172.21.15.49:36836) 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbf7fe640 1 -- 172.21.15.49:0/3615753257 learned_addr learned my addr 172.21.15.49:0/3615753257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc486e640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c12d30 0x55ee29c174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 -- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c12d30 msgr2=0x55ee29c174f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c12d30 0x55ee29c174f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 -- 172.21.15.49:0/3615753257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 msgr2=0x55ee29c16fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 -- 172.21.15.49:0/3615753257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee29c886c0 con 0x55ee29c09830 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbbffff640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 0x55ee29c16a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0030c90 tx=0x7fbbb0030d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1ee7737162742a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb003f030 con 0x55ee29c09830 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fbbb0009040 con 0x55ee29c09830 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.342+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee29b74d20 con 0x55ee29c09830 2024-03-16T08:23:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbbb0049050 con 0x55ee29c09830 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 10 client.26864 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ee29c899a0 con 0x55ee29c09830 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ee29b74d20 con 0x55ee29c09830 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbbb0004b20 con 0x55ee29c09830 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 10 client.26864 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 10 client.26864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 20 client.26864 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c8a000 0x55ee29c8c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 4 client.26864 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee29af26a0 con 0x55ee29c8a000 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c8a000 0x55ee29c8c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbb9f7fe640 10 client.26864 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.343+0000 7fbbbf7fe640 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c8a000 0x55ee29c8c3f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0a23f5735532a3b server_cookie=9c885eef88026bd6 in_seq=0 out_seq=0 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbb9f7fe640 1 -- 172.21.15.49:0/3615753257 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee29af26a0 con 0x55ee29c8a000 2024-03-16T08:23:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbb9f7fe640 10 client.26864 handle_command_reply: tid=0 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 client.26864 shutdown 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 msgr2=0x55ee29c16a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 0x55ee29c16a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0030c90 tx=0x7fbbb0030d60 comp rx=0 tx=0).stop 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 shutdown_connections 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55ee29c8a000 0x55ee29c8c3f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee29c12d30 0x55ee29c174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee29c0a150 0x55ee29c16fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 --2- 172.21.15.49:0/3615753257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee29c09830 0x55ee29c16a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbb9f7fe640 0 client.26864 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 >> 172.21.15.49:0/3615753257 conn(0x55ee29c03f60 msgr2=0x55ee29c02fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 shutdown_connections 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 1 -- 172.21.15.49:0/3615753257 wait complete. 2024-03-16T08:23:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc8f99740 20 client.26864 trim_cache size 0 max 16384 2024-03-16T08:23:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:01.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 msgr2=0x7fbba80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 0x7fbba80a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009b50 tx=0x7fbbb0005f90 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 msgr2=0x7fbbc019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.344+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800eb10 tx=0x7fbbb800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 shutdown_connections 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fbba80a2970 0x7fbba80a4e30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbc0107790 0x7fbbc019c950 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbc0102140 0x7fbbc0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 --2- 172.21.15.49:0/1654030847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc0101780 0x7fbbc01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 >> 172.21.15.49:0/1654030847 conn(0x7fbbc00fd210 msgr2=0x7fbbc00fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 shutdown_connections 2024-03-16T08:23:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.345+0000 7fbbc62d1640 1 -- 172.21.15.49:0/1654030847 wait complete. 2024-03-16T08:23:01.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:23:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 -- 172.21.15.49:0/2315886357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 msgr2=0x7fcd74105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2315886357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd74105620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009890 tx=0x7fcd6402bc00 comp rx=0 tx=0).stop 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 -- 172.21.15.49:0/2315886357 shutdown_connections 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2315886357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74105c40 0x7fcd741060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2315886357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd74105620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2315886357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74100840 0x7fcd74100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.465+0000 7fcd7b509640 1 -- 172.21.15.49:0/2315886357 >> 172.21.15.49:0/2315886357 conn(0x7fcd74077840 msgr2=0x7fcd74077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- 172.21.15.49:0/2315886357 shutdown_connections 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- 172.21.15.49:0/2315886357 wait complete. 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 Processor -- start 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- start start 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74100840 0x7fcd7419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd7419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741181c0 con 0x7fcd74105c40 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74118040 con 0x7fcd74101160 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd7b509640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74118340 con 0x7fcd74100840 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd79aa6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd79aa6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36854/0 (socket says 172.21.15.49:36854) 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd79aa6640 1 -- 172.21.15.49:0/2106941248 learned_addr learned my addr 172.21.15.49:0/2106941248 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.466+0000 7fcd78aa4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd7419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd792a5640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74100840 0x7fcd7419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 -- 172.21.15.49:0/2106941248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74100840 msgr2=0x7fcd7419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74100840 0x7fcd7419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 -- 172.21.15.49:0/2106941248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 msgr2=0x7fcd7419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd7419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 -- 172.21.15.49:0/2106941248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd741a36b0 con 0x7fcd74105c40 2024-03-16T08:23:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd7419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd79aa6640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00eb10 tx=0x7fcd6c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd6a7fc640 1 -- 172.21.15.49:0/2106941248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00ce20 con 0x7fcd74105c40 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd741a39a0 con 0x7fcd74105c40 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd6a7fc640 1 -- 172.21.15.49:0/2106941248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6c005020 con 0x7fcd74105c40 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd6a7fc640 1 -- 172.21.15.49:0/2106941248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c010690 con 0x7fcd74105c40 2024-03-16T08:23:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.467+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741ab230 con 0x7fcd74105c40 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd6a7fc640 1 -- 172.21.15.49:0/2106941248 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fcd6c005540 con 0x7fcd74105c40 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 Processor -- start 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 -- start start 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff78ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aff7847490 con 0x55aff78e4150 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aff77efc90 con 0x55aff78e3830 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55aff7843530 con 0x55aff78ecd30 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36858/0 (socket says 172.21.15.49:36858) 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd792a5640 1 -- 172.21.15.49:0/425173253 learned_addr learned my addr 172.21.15.49:0/425173253 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff78ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd6a7fc640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 0x7fcd600a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.470+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd79aa6640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 0x7fcd600a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 -- 172.21.15.49:0/425173253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 msgr2=0x55aff78ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff78ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd6a7fc640 1 -- 172.21.15.49:0/2106941248 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcd6c0cf310 con 0x7fcd74105c40 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 -- 172.21.15.49:0/425173253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 msgr2=0x55aff78e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 -- 172.21.15.49:0/425173253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff78aa500 con 0x55aff78e4150 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff78ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd792a5640 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009cd0 tx=0x7fcd70011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5113e42f9d859dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd79aa6640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 0x7fcd600a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c09b6c0 tx=0x7fcd6c086040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd5f7fe640 1 -- 172.21.15.49:0/425173253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001e070 con 0x55aff78e4150 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd5f7fe640 1 -- 172.21.15.49:0/425173253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd70019b10 con 0x55aff78e4150 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.471+0000 7fcd5f7fe640 1 -- 172.21.15.49:0/425173253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000ea90 con 0x55aff78e4150 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/425173253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 msgr2=0x55aff78ec7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009cd0 tx=0x7fcd70011810 comp rx=0 tx=0).stop 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/425173253 shutdown_connections 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff78ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ec7f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/425173253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/425173253 >> 172.21.15.49:0/425173253 conn(0x55aff78ddf60 msgr2=0x55aff78de340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/425173253 shutdown_connections 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/425173253 wait complete. 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 -- start start 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.472+0000 7fcd7e1d5740 1 Processor -- start 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e3830 0x55aff78e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78ecd30 0x55aff78ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aff78efb20 con 0x55aff78e4150 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aff7803230 con 0x55aff78ecd30 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55aff78efca0 con 0x55aff78e3830 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 10 client.0 _ll_register_callbacks cb 0x55aff783f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36870/0 (socket says 172.21.15.49:36870) 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 -- 172.21.15.49:0/2718117638 learned_addr learned my addr 172.21.15.49:0/2718117638 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78ecd30 0x55aff78ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e3830 0x55aff78e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 -- 172.21.15.49:0/2718117638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e3830 msgr2=0x55aff78e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e3830 0x55aff78e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 -- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78ecd30 msgr2=0x55aff78ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78ecd30 0x55aff78ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 -- 172.21.15.49:0/2718117638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff7812100 con 0x55aff78e4150 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd792a5640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005890 tx=0x7fcd70009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1417d630030d94 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001e050 con 0x55aff78e4150 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcd70005450 con 0x55aff78e4150 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.473+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd70019b10 con 0x55aff78e4150 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd70033050 con 0x55aff78e4150 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 10 client.16995 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aff755b3f0 con 0x55aff78e4150 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd70019b10 con 0x55aff78e4150 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcd700046c0 con 0x55aff78e4150 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 10 client.16995 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 20 client.16995 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff78fd000 0x55aff78ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 4 client.16995 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aff78549d0 con 0x55aff78fd000 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.474+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff78fd000 0x55aff78ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff78fd000 0x55aff78ff3f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb224ea1262b32ec server_cookie=89fe019c5e41e5d4 in_seq=0 out_seq=0 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd5cff9640 10 client.16995 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd5cff9640 1 -- 172.21.15.49:0/2718117638 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aff78549d0 con 0x55aff78fd000 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd5cff9640 10 client.16995 handle_command_reply: tid=0 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 client.16995 shutdown 2024-03-16T08:23:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 msgr2=0x55aff78ea300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005890 tx=0x7fcd70009720 comp rx=0 tx=0).stop 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 shutdown_connections 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff78fd000 0x55aff78ff3f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78ecd30 0x55aff78ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd5cff9640 0 client.16995 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78ea300 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/2718117638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e3830 0x55aff78e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.475+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 >> 172.21.15.49:0/2718117638 conn(0x55aff78ddf60 msgr2=0x55aff78dfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.476+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 shutdown_connections 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.476+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/2718117638 wait complete. 2024-03-16T08:23:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.476+0000 7fcd7e1d5740 20 client.16995 trim_cache size 0 max 16384 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 Processor -- start 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 -- start start 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff7904b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b62a0 con 0x55aff78e4150 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b3b70 con 0x55aff78e3830 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b19a0 con 0x55aff78ecd30 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36872/0 (socket says 172.21.15.49:36872) 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 -- 172.21.15.49:0/721029320 learned_addr learned my addr 172.21.15.49:0/721029320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff7904b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 -- 172.21.15.49:0/721029320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 msgr2=0x55aff7904b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff7904b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 -- 172.21.15.49:0/721029320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 msgr2=0x55aff78ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78ec4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.489+0000 7fcd792a5640 1 -- 172.21.15.49:0/721029320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff78e3f80 con 0x55aff78e4150 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd792a5640 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcd70025450 tx=0x7fcd70011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5755f5f63bde55e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd5effd640 1 -- 172.21.15.49:0/721029320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001e070 con 0x55aff78e4150 2024-03-16T08:23:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd5effd640 1 -- 172.21.15.49:0/721029320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd7002a070 con 0x55aff78e4150 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd5effd640 1 -- 172.21.15.49:0/721029320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70009920 con 0x55aff78e4150 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/721029320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 msgr2=0x55aff78f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcd70025450 tx=0x7fcd70011dc0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/721029320 shutdown_connections 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78ecd30 0x55aff7904b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78e4150 0x55aff78f88c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/721029320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/721029320 >> 172.21.15.49:0/721029320 conn(0x55aff78ddf60 msgr2=0x55aff78e50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/721029320 shutdown_connections 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.490+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/721029320 wait complete. 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 -- start start 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 Processor -- start 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e4150 0x55aff78e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b2030 con 0x55aff78ecd30 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b34a0 con 0x55aff78e3830 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b2700 con 0x55aff78e4150 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd7e1d5740 10 client.0 _ll_register_callbacks cb 0x7fcd7001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36874/0 (socket says 172.21.15.49:36874) 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 -- 172.21.15.49:0/4259750383 learned_addr learned my addr 172.21.15.49:0/4259750383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd792a5640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e4150 0x55aff78e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 -- 172.21.15.49:0/4259750383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e4150 msgr2=0x55aff78e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e4150 0x55aff78e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 -- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 msgr2=0x55aff78e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.491+0000 7fcd78aa4640 1 -- 172.21.15.49:0/4259750383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff76257d0 con 0x55aff78ecd30 2024-03-16T08:23:01.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd78aa4640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcd6402e010 tx=0x7fcd7000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf6d119825a34dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64039040 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fcd6404f2c0 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd7002a070 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd6404f4c0 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 10 client.17001 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aff79632a0 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd7002a070 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcd6403f040 con 0x55aff78ecd30 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 10 client.17001 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 20 client.17001 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff7963920 0x55aff7965d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:01.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 4 client.17001 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.492+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aff784ed20 con 0x55aff7963920 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd792a5640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff7963920 0x55aff7965d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd792a5640 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff7963920 0x55aff7965d10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b795c81ab6cb86b server_cookie=b30b655116268142 in_seq=0 out_seq=0 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd5cff9640 10 client.17001 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd5cff9640 1 -- 172.21.15.49:0/4259750383 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aff784ed20 con 0x55aff7963920 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd5cff9640 10 client.17001 handle_command_reply: tid=0 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 client.17001 shutdown 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 msgr2=0x55aff78e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcd6402e010 tx=0x7fcd7000e5f0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 shutdown_connections 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55aff7963920 0x55aff7965d10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aff78ecd30 0x55aff78e73c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aff78e4150 0x55aff78e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 --2- 172.21.15.49:0/4259750383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aff78e3830 0x55aff78e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd5cff9640 0 client.17001 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 >> 172.21.15.49:0/4259750383 conn(0x55aff78ddf60 msgr2=0x55aff78e9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 shutdown_connections 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 1 -- 172.21.15.49:0/4259750383 wait complete. 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.493+0000 7fcd7e1d5740 20 client.17001 trim_cache size 0 max 16384 2024-03-16T08:23:01.500 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 msgr2=0x7fcd600a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 0x7fcd600a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c09b6c0 tx=0x7fcd6c086040 comp rx=0 tx=0).stop 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 msgr2=0x7fcd741a2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00eb10 tx=0x7fcd6c00ef10 comp rx=0 tx=0).stop 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 shutdown_connections 2024-03-16T08:23:01.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fcd600a2970 0x7fcd600a4e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74105c40 0x7fcd741a2f80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd74101160 0x7fcd7419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 --2- 172.21.15.49:0/2106941248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd74100840 0x7fcd7419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 >> 172.21.15.49:0/2106941248 conn(0x7fcd74077840 msgr2=0x7fcd741035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 shutdown_connections 2024-03-16T08:23:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:01.494+0000 7fcd7b509640 1 -- 172.21.15.49:0/2106941248 wait complete. 2024-03-16T08:23:01.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1070397608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1070397608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:01.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1070397608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:02.506 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:23:02.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:23:02.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 -- 172.21.15.49:0/2151584109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370103370 msgr2=0x7f737010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 --2- 172.21.15.49:0/2151584109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370103370 0x7f737010b850 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f736c009900 tx=0x7f736c02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 -- 172.21.15.49:0/2151584109 shutdown_connections 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 --2- 172.21.15.49:0/2151584109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f737010be20 0x7f737010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 --2- 172.21.15.49:0/2151584109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370103370 0x7f737010b850 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 --2- 172.21.15.49:0/2151584109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7370102a50 0x7f7370102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 -- 172.21.15.49:0/2151584109 >> 172.21.15.49:0/2151584109 conn(0x7f7370077840 msgr2=0x7f7370077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 -- 172.21.15.49:0/2151584109 shutdown_connections 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.630+0000 7f7377ab2640 1 -- 172.21.15.49:0/2151584109 wait complete. 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 Processor -- start 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 -- start start 2024-03-16T08:23:02.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7370103370 0x7f73701986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 0x7f737019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7370111bf0 con 0x7f7370102a50 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7370111a70 con 0x7f7370103370 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f7377ab2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7370111d70 con 0x7f737010be20 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39452/0 (socket says 172.21.15.49:39452) 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 -- 172.21.15.49:0/3530124953 learned_addr learned my addr 172.21.15.49:0/3530124953 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737504d640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7370103370 0x7f73701986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737604f640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 0x7f737019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 -- 172.21.15.49:0/3530124953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 msgr2=0x7f737019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 0x7f737019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 -- 172.21.15.49:0/3530124953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7370103370 msgr2=0x7f73701986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7370103370 0x7f73701986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 -- 172.21.15.49:0/3530124953 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737019d130 con 0x7f7370102a50 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737604f640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 0x7f737019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.631+0000 7f737584e640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9f0 tx=0x7f736400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.632+0000 7f7362ffd640 1 -- 172.21.15.49:0/3530124953 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736400cd80 con 0x7f7370102a50 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.632+0000 7f7362ffd640 1 -- 172.21.15.49:0/3530124953 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7364005500 con 0x7f7370102a50 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.632+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737019d420 con 0x7f7370102a50 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.632+0000 7f7362ffd640 1 -- 172.21.15.49:0/3530124953 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364010690 con 0x7f7370102a50 2024-03-16T08:23:02.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.632+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701a4cb0 con 0x7f7370102a50 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.634+0000 7f7362ffd640 1 -- 172.21.15.49:0/3530124953 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f73640046c0 con 0x7f7370102a50 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.634+0000 7f737a77f740 1 Processor -- start 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 -- start start 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676e710 0x5584c676d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5584c66d10d0 con 0x5584c676da30 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584c6679c70 con 0x5584c677b1b0 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737a77f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584c66cd170 con 0x5584c676e710 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:33206/0 (socket says 172.21.15.49:33206) 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 -- 172.21.15.49:0/2970957429 learned_addr learned my addr 172.21.15.49:0/2970957429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39464/0 (socket says 172.21.15.49:39464) 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737604f640 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676e710 0x5584c676d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f7362ffd640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 0x7f735c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 -- 172.21.15.49:0/2970957429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676e710 msgr2=0x5584c676d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676e710 0x5584c676d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 -- 172.21.15.49:0/2970957429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 msgr2=0x5584c676deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737504d640 1 -- 172.21.15.49:0/2970957429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c64ad7e0 con 0x5584c677b1b0 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737584e640 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f737604f640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 0x7f735c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.635+0000 7f7362ffd640 1 -- 172.21.15.49:0/3530124953 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f73640cf2b0 con 0x7f7370102a50 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f737504d640 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f736c009230 tx=0x7f736c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f66b864640d7ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f735bfff640 1 -- 172.21.15.49:0/2970957429 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c042070 con 0x5584c677b1b0 2024-03-16T08:23:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f737604f640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 0x7f735c0a4e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7368005fd0 tx=0x7f7368014040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f735bfff640 1 -- 172.21.15.49:0/2970957429 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736c004040 con 0x5584c677b1b0 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f735bfff640 1 -- 172.21.15.49:0/2970957429 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c004c60 con 0x5584c677b1b0 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f737a77f740 1 -- 172.21.15.49:0/2970957429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 msgr2=0x5584c677d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.636+0000 7f737a77f740 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f736c009230 tx=0x7f736c0095f0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 -- 172.21.15.49:0/2970957429 shutdown_connections 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c677b1b0 0x5584c677d5a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c676deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 --2- 172.21.15.49:0/2970957429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676e710 0x5584c676d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 -- 172.21.15.49:0/2970957429 >> 172.21.15.49:0/2970957429 conn(0x5584c6734c50 msgr2=0x5584c6768480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 -- 172.21.15.49:0/2970957429 shutdown_connections 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 -- 172.21.15.49:0/2970957429 wait complete. 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 -- start start 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.637+0000 7f737a77f740 1 Processor -- start 2024-03-16T08:23:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 0x5584c67780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6778610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5584c677dfc0 con 0x5584c677b1b0 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584c668d0d0 con 0x5584c676e710 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584c677e140 con 0x5584c676da30 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737a77f740 10 client.0 _ll_register_callbacks cb 0x5584c6656760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737584e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 0x5584c67780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737504d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6778610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39466/0 (socket says 172.21.15.49:39466) 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 -- 172.21.15.49:0/1266625600 learned_addr learned my addr 172.21.15.49:0/1266625600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 -- 172.21.15.49:0/1266625600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 msgr2=0x5584c67780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 0x5584c67780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 -- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 msgr2=0x5584c6778610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6778610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 -- 172.21.15.49:0/1266625600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c650df70 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737504d640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6778610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737584e640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 0x5584c67780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f737604f640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7368011a90 tx=0x7f7368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0ead76f304d98a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f73597fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.638+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736802b060 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f736801bdc0 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736c004040 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7368030460 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 10 client.17013 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5584c63a5170 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736c004040 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f736800f7b0 con 0x5584c677b1b0 2024-03-16T08:23:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 10 client.17013 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 20 client.17013 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c678b330 0x5584c678d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 4 client.17013 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c66de610 con 0x5584c678b330 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.639+0000 7f737584e640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c678b330 0x5584c678d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.640+0000 7f737584e640 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c678b330 0x5584c678d780 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3808caabb241567f server_cookie=a9e1d107a93ffe1d in_seq=0 out_seq=0 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.640+0000 7f73597fa640 10 client.17013 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.640+0000 7f73597fa640 1 -- 172.21.15.49:0/1266625600 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5584c66de610 con 0x5584c678b330 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.640+0000 7f73597fa640 10 client.17013 handle_command_reply: tid=0 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.640+0000 7f737a77f740 1 client.17013 shutdown 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 msgr2=0x5584c6778b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7368011a90 tx=0x7f7368004a40 comp rx=0 tx=0).stop 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 shutdown_connections 2024-03-16T08:23:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c678b330 0x5584c678d780 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c677b1b0 0x5584c6778b50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6778610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 --2- 172.21.15.49:0/1266625600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c676da30 0x5584c67780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 >> 172.21.15.49:0/1266625600 conn(0x5584c6734c50 msgr2=0x5584c676a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f73597fa640 0 client.17013 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 shutdown_connections 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 1 -- 172.21.15.49:0/1266625600 wait complete. 2024-03-16T08:23:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.641+0000 7f737a77f740 20 client.17013 trim_cache size 0 max 16384 2024-03-16T08:23:02.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 Processor -- start 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 -- start start 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6786e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6793090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73701ad9a0 con 0x5584c676da30 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73701ab270 con 0x5584c676e710 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737a77f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a90a0 con 0x5584c677b1b0 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39472/0 (socket says 172.21.15.49:39472) 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 -- 172.21.15.49:0/2313412483 learned_addr learned my addr 172.21.15.49:0/2313412483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737604f640 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6793090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737504d640 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6786e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 -- 172.21.15.49:0/2313412483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 msgr2=0x5584c6793090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6793090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 -- 172.21.15.49:0/2313412483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 msgr2=0x5584c6786e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6786e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 -- 172.21.15.49:0/2313412483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c6789e30 con 0x5584c676da30 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.659+0000 7f737584e640 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9c0 tx=0x7f7364086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc895a86c934a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f735b7fe640 1 -- 172.21.15.49:0/2313412483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736409b4a0 con 0x5584c676da30 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f735b7fe640 1 -- 172.21.15.49:0/2313412483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736409c060 con 0x5584c676da30 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f735b7fe640 1 -- 172.21.15.49:0/2313412483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736409b2c0 con 0x5584c676da30 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- 172.21.15.49:0/2313412483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 msgr2=0x5584c67779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9c0 tx=0x7f7364086040 comp rx=0 tx=0).stop 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- 172.21.15.49:0/2313412483 shutdown_connections 2024-03-16T08:23:02.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6793090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676e710 0x5584c6786e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 --2- 172.21.15.49:0/2313412483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676da30 0x5584c67779b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- 172.21.15.49:0/2313412483 >> 172.21.15.49:0/2313412483 conn(0x5584c6734c50 msgr2=0x5584c677a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- 172.21.15.49:0/2313412483 shutdown_connections 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- 172.21.15.49:0/2313412483 wait complete. 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.660+0000 7f737a77f740 1 -- start start 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 Processor -- start 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676da30 0x5584c6774d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6775780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a9730 con 0x5584c676e710 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73701aaba0 con 0x5584c676da30 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a9e00 con 0x5584c677b1b0 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737a77f740 10 client.0 _ll_register_callbacks cb 0x7f7368009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:02.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39474/0 (socket says 172.21.15.49:39474) 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737584e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676da30 0x5584c6774d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 -- 172.21.15.49:0/2835560744 learned_addr learned my addr 172.21.15.49:0/2835560744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737604f640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6775780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 -- 172.21.15.49:0/2835560744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 msgr2=0x5584c6775780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6775780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 -- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676da30 msgr2=0x5584c6774d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676da30 0x5584c6774d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.661+0000 7f737504d640 1 -- 172.21.15.49:0/2835560744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c676ca60 con 0x5584c676e710 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737504d640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f736c005bc0 tx=0x7f736c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfa1e519dd48f55 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c055060 con 0x5584c676e710 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f736c004ad0 con 0x5584c676e710 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736409c060 con 0x5584c676e710 2024-03-16T08:23:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f736c042040 con 0x5584c676e710 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 10 client.17019 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5584c67f18d0 con 0x5584c676e710 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736409c060 con 0x5584c676e710 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f736c05a560 con 0x5584c676e710 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 10 client.17019 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 20 client.17019 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c67f1c10 0x5584c67f4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 4 client.17019 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.662+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5584c66d8960 con 0x5584c67f1c10 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.663+0000 7f737584e640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c67f1c10 0x5584c67f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.663+0000 7f737584e640 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c67f1c10 0x5584c67f4030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c5d0468ec3035c server_cookie=b8738fc3f3fba50c in_seq=0 out_seq=0 2024-03-16T08:23:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.663+0000 7f73597fa640 10 client.17019 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f73597fa640 1 -- 172.21.15.49:0/2835560744 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5584c66d8960 con 0x5584c67f1c10 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f73597fa640 10 client.17019 handle_command_reply: tid=0 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 client.17019 shutdown 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 msgr2=0x5584c6775240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f736c005bc0 tx=0x7f736c005b50 comp rx=0 tx=0).stop 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 shutdown_connections 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5584c67f1c10 0x5584c67f4030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584c677b1b0 0x5584c6775780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584c676e710 0x5584c6775240 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f73597fa640 0 client.17019 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 --2- 172.21.15.49:0/2835560744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584c676da30 0x5584c6774d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.664+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 >> 172.21.15.49:0/2835560744 conn(0x5584c6734c50 msgr2=0x5584c6766c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 shutdown_connections 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f737a77f740 1 -- 172.21.15.49:0/2835560744 wait complete. 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f737a77f740 20 client.17019 trim_cache size 0 max 16384 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "cca95594-cada-4f79-8ec5-4e2c15dcd4f7", 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:23:02.671 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 msgr2=0x7f735c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 0x7f735c0a4e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7368005fd0 tx=0x7f7368014040 comp rx=0 tx=0).stop 2024-03-16T08:23:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 msgr2=0x7f7370198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9f0 tx=0x7f736400edf0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 shutdown_connections 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f735c0a2970 0x7f735c0a4e30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f737010be20 0x7f737019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7370103370 0x7f73701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 --2- 172.21.15.49:0/3530124953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7370102a50 0x7f7370198190 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 >> 172.21.15.49:0/3530124953 conn(0x7f7370077840 msgr2=0x7f73700fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 shutdown_connections 2024-03-16T08:23:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.665+0000 7f7377ab2640 1 -- 172.21.15.49:0/3530124953 wait complete. 2024-03-16T08:23:02.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:23:02.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:23:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 -- 172.21.15.49:0/38453474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c102a50 msgr2=0x7fe21c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 --2- 172.21.15.49:0/38453474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c102a50 0x7fe21c102e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe214009900 tx=0x7fe21402bcd0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 -- 172.21.15.49:0/38453474 shutdown_connections 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 --2- 172.21.15.49:0/38453474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 0x7fe21c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 --2- 172.21.15.49:0/38453474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c103370 0x7fe21c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 --2- 172.21.15.49:0/38453474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c102a50 0x7fe21c102e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 -- 172.21.15.49:0/38453474 >> 172.21.15.49:0/38453474 conn(0x7fe21c077840 msgr2=0x7fe21c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 -- 172.21.15.49:0/38453474 shutdown_connections 2024-03-16T08:23:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 -- 172.21.15.49:0/38453474 wait complete. 2024-03-16T08:23:02.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:02 smithi049 ceph-mon[31942]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.787+0000 7fe2215a9640 1 Processor -- start 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 -- start start 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c103370 0x7fe21c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 0x7fe21c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c111bf0 con 0x7fe21c102a50 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c111a70 con 0x7fe21c103370 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe2215a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c111d70 con 0x7fe21c10be20 2024-03-16T08:23:02.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39502/0 (socket says 172.21.15.49:39502) 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 -- 172.21.15.49:0/399231363 learned_addr learned my addr 172.21.15.49:0/399231363 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 0x7fe21c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c103370 0x7fe21c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 -- 172.21.15.49:0/399231363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 msgr2=0x7fe21c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 0x7fe21c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 -- 172.21.15.49:0/399231363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c103370 msgr2=0x7fe21c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c103370 0x7fe21c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 -- 172.21.15.49:0/399231363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c19d0b0 con 0x7fe21c102a50 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe21affd640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe214005940 tx=0x7fe21402cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.788+0000 7fe203fff640 1 -- 172.21.15.49:0/399231363 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214041070 con 0x7fe21c102a50 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.789+0000 7fe203fff640 1 -- 172.21.15.49:0/399231363 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe214004450 con 0x7fe21c102a50 2024-03-16T08:23:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.789+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c19d340 con 0x7fe21c102a50 2024-03-16T08:23:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.789+0000 7fe203fff640 1 -- 172.21.15.49:0/399231363 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214003d80 con 0x7fe21c102a50 2024-03-16T08:23:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.789+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c1a4c30 con 0x7fe21c102a50 2024-03-16T08:23:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe203fff640 1 -- 172.21.15.49:0/399231363 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fe214044070 con 0x7fe21c102a50 2024-03-16T08:23:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 Processor -- start 2024-03-16T08:23:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 -- start start 2024-03-16T08:23:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe203fff640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 0x7fe2040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e406aef500 con 0x55e406b8b150 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e406a97be0 con 0x55e406b94650 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe224194740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e406aeb5a0 con 0x55e406b8ba70 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.791+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39514/0 (socket says 172.21.15.49:39514) 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 0x7fe2040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4093801959 learned_addr learned my addr 172.21.15.49:0/4093801959 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:37564/0 (socket says 172.21.15.49:37564) 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe203fff640 1 -- 172.21.15.49:0/399231363 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe2140f07b0 con 0x7fe21c102a50 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4093801959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 msgr2=0x55e406b94110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4093801959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 msgr2=0x55e406b96a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b96a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4093801959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e406aab4b0 con 0x55e406b8b150 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21affd640 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0099e0 tx=0x7fe20c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b017fc6b2ddfe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 0x7fe2040a4e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2100095f0 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe2017fa640 1 -- 172.21.15.49:0/4093801959 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c004b20 con 0x55e406b8b150 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe2017fa640 1 -- 172.21.15.49:0/4093801959 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c011d80 con 0x55e406b8b150 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.792+0000 7fe2017fa640 1 -- 172.21.15.49:0/4093801959 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005730 con 0x55e406b8b150 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- 172.21.15.49:0/4093801959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 msgr2=0x55e406b8b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0099e0 tx=0x7fe20c009500 comp rx=0 tx=0).stop 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- 172.21.15.49:0/4093801959 shutdown_connections 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b96a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 --2- 172.21.15.49:0/4093801959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b8b530 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- 172.21.15.49:0/4093801959 >> 172.21.15.49:0/4093801959 conn(0x55e406b85880 msgr2=0x55e406b85c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- 172.21.15.49:0/4093801959 shutdown_connections 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- 172.21.15.49:0/4093801959 wait complete. 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 -- start start 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.793+0000 7fe224194740 1 Processor -- start 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8b150 0x55e406b91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e406b973e0 con 0x55e406b94650 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e406aab160 con 0x55e406b8ba70 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e406b97560 con 0x55e406b8b150 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 10 client.0 _ll_register_callbacks cb 0x55e406a8f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39518/0 (socket says 172.21.15.49:39518) 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8b150 0x55e406b91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4125734134 learned_addr learned my addr 172.21.15.49:0/4125734134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4125734134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8b150 msgr2=0x55e406b91650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8b150 0x55e406b91650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 msgr2=0x55e406b91b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b91b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4125734134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4068cd7d0 con 0x55e406b94650 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3eb0fffeb25b7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe1feff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c003d80 con 0x55e406b94650 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe20c005470 con 0x55e406b94650 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.794+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c011d80 con 0x55e406b94650 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe20c02d020 con 0x55e406b94650 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 10 client.17031 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4068033f0 con 0x55e406b94650 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c011d80 con 0x55e406b94650 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe20c0041e0 con 0x55e406b94650 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 10 client.17031 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 20 client.17031 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406ba4440 0x55e406ba6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 4 client.17031 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e406afca40 con 0x55e406ba4440 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.795+0000 7fe21affd640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406ba4440 0x55e406ba6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe1feff5640 10 client.17031 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe21affd640 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406ba4440 0x55e406ba6830 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25fc41a010808283 server_cookie=91e69aa62dce072f in_seq=0 out_seq=0 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe1feff5640 1 -- 172.21.15.49:0/4125734134 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e406afca40 con 0x55e406ba4440 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe1feff5640 10 client.17031 handle_command_reply: tid=0 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 client.17031 shutdown 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 msgr2=0x55e406b920d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c004cf0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 shutdown_connections 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406ba4440 0x55e406ba6830 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b94650 0x55e406b920d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b91b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 --2- 172.21.15.49:0/4125734134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8b150 0x55e406b91650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe1feff5640 0 client.17031 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.796+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 >> 172.21.15.49:0/4125734134 conn(0x55e406b85880 msgr2=0x55e406b875d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.797+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 shutdown_connections 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.797+0000 7fe224194740 1 -- 172.21.15.49:0/4125734134 wait complete. 2024-03-16T08:23:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.797+0000 7fe224194740 20 client.17031 trim_cache size 0 max 16384 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 Processor -- start 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 -- start start 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b90e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b94650 0x55e406bac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1afcd0 con 0x55e406b8b150 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1ad5a0 con 0x55e406b8ba70 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe224194740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1ab3d0 con 0x55e406b94650 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39526/0 (socket says 172.21.15.49:39526) 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe21affd640 1 -- 172.21.15.49:0/738718454 learned_addr learned my addr 172.21.15.49:0/738718454 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b90e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.814+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b94650 0x55e406bac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 -- 172.21.15.49:0/738718454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b94650 msgr2=0x55e406bac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b94650 0x55e406bac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 -- 172.21.15.49:0/738718454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 msgr2=0x55e406b90e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b90e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 -- 172.21.15.49:0/738718454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4067c30f0 con 0x55e406b8b150 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe21affd640 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe21402c580 tx=0x7fe21402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0d96e1e623f2ca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe200ff9640 1 -- 172.21.15.49:0/738718454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21404b040 con 0x55e406b8b150 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe200ff9640 1 -- 172.21.15.49:0/738718454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe214004c80 con 0x55e406b8b150 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe200ff9640 1 -- 172.21.15.49:0/738718454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140c5710 con 0x55e406b8b150 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 -- 172.21.15.49:0/738718454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 msgr2=0x55e406ba0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe21402c580 tx=0x7fe21402bcd0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 -- 172.21.15.49:0/738718454 shutdown_connections 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b94650 0x55e406bac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b8ba70 0x55e406b90e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 --2- 172.21.15.49:0/738718454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406ba0180 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 -- 172.21.15.49:0/738718454 >> 172.21.15.49:0/738718454 conn(0x55e406b85880 msgr2=0x55e406b91ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.815+0000 7fe224194740 1 -- 172.21.15.49:0/738718454 shutdown_connections 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 -- 172.21.15.49:0/738718454 wait complete. 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 -- start start 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 Processor -- start 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b98ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1aba60 con 0x55e406b8b150 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1aced0 con 0x55e406b94650 2024-03-16T08:23:02.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c1ac130 con 0x55e406b8ba70 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe224194740 10 client.0 _ll_register_callbacks cb 0x7fe20c0135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39528/0 (socket says 172.21.15.49:39528) 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b98ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21affd640 1 -- 172.21.15.49:0/4058816281 learned_addr learned my addr 172.21.15.49:0/4058816281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4058816281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 msgr2=0x55e406b985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.816+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4058816281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 msgr2=0x55e406b98060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21b7fe640 1 -- 172.21.15.49:0/4058816281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e406ab69d0 con 0x55e406b94650 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21a7fc640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21affd640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe21b7fe640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b98ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009270 tx=0x7fe20c004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed19c0bdb716749e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005bd0 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fe20c013250 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214004c80 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe20c003b80 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 10 client.26918 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e406c0a7a0 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe214004c80 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe20c032630 con 0x55e406b94650 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 10 client.26918 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 20 client.26918 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:02.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406c0af10 0x55e406c0d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 4 client.26918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.817+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e406af6d90 con 0x55e406c0af10 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe21affd640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406c0af10 0x55e406c0d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe1feff5640 10 client.26918 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe21affd640 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406c0af10 0x55e406c0d330 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a12db428fa8c1e5 server_cookie=69cbcc605e802a29 in_seq=0 out_seq=0 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe1feff5640 1 -- 172.21.15.49:0/4058816281 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e406af6d90 con 0x55e406c0af10 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe1feff5640 10 client.26918 handle_command_reply: tid=0 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 client.26918 shutdown 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 msgr2=0x55e406b98ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b98ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009270 tx=0x7fe20c004840 comp rx=0 tx=0).stop 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 shutdown_connections 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e406c0af10 0x55e406c0d330 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e406b94650 0x55e406b98ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe1feff5640 0 client.26918 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e406b8ba70 0x55e406b985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 --2- 172.21.15.49:0/4058816281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e406b8b150 0x55e406b98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 >> 172.21.15.49:0/4058816281 conn(0x55e406b85880 msgr2=0x55e406b84070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 shutdown_connections 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 1 -- 172.21.15.49:0/4058816281 wait complete. 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.818+0000 7fe224194740 20 client.26918 trim_cache size 0 max 16384 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "cca95594-cada-4f79-8ec5-4e2c15dcd4f7": { 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "cca95594-cada-4f79-8ec5-4e2c15dcd4f7", 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:23:02.828 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:23:02.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:23:02.829 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 msgr2=0x7fe2040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 0x7fe2040a4e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2100095f0 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 msgr2=0x7fe21c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe214005940 tx=0x7fe21402cbd0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 shutdown_connections 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe2040a2970 0x7fe2040a4e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21c10be20 0x7fe21c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe21c103370 0x7fe21c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 --2- 172.21.15.49:0/399231363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe21c102a50 0x7fe21c198110 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:02.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 >> 172.21.15.49:0/399231363 conn(0x7fe21c077840 msgr2=0x7fe21c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:02.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 shutdown_connections 2024-03-16T08:23:02.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:02.819+0000 7fe2215a9640 1 -- 172.21.15.49:0/399231363 wait complete. 2024-03-16T08:23:02.831 INFO:tasks.cephfs.filesystem:scrub status for tag:cca95594-cada-4f79-8ec5-4e2c15dcd4f7 - {'path': '/', 'tag': 'cca95594-cada-4f79-8ec5-4e2c15dcd4f7', 'options': 'recursive,force'} 2024-03-16T08:23:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:02 smithi129 ceph-mon[36561]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:02.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:02 smithi154 ceph-mon[36471]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:03.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:03 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:23:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:03 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:03 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:23:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:03 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:03.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:03 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:23:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:03 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:03 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:23:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:03 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:04.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:03 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:23:04.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:03 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:04.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:03 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:23:04.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:03 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:04 smithi129 ceph-mon[36561]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:04.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:04 smithi154 ceph-mon[36471]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:05.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:04 smithi049 ceph-mon[31942]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:06.840 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:06 smithi049 ceph-mon[31942]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:06.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:06 smithi129 ceph-mon[36561]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:06.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:06 smithi154 ceph-mon[36471]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:07.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:07 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:23:07.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:07 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:23:08.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:07 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:23:08.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:08 smithi129 ceph-mon[36561]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:08.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:08 smithi154 ceph-mon[36471]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:09.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:08 smithi049 ceph-mon[31942]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:10.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:10 smithi129 ceph-mon[36561]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:10.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:10 smithi154 ceph-mon[36471]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:11.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:10 smithi049 ceph-mon[31942]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:12.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:12 smithi129 ceph-mon[36561]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:12.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:12 smithi154 ceph-mon[36471]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:13.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:12 smithi049 ceph-mon[31942]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:14.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:14 smithi129 ceph-mon[36561]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:14.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:14 smithi154 ceph-mon[36471]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:15.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:14 smithi049 ceph-mon[31942]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-16T08:23:16.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:16 smithi129 ceph-mon[36561]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:16.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:16 smithi154 ceph-mon[36471]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:17.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:16 smithi049 ceph-mon[31942]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-16T08:23:18.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:18 smithi129 ceph-mon[36561]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:23:18.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:18 smithi154 ceph-mon[36471]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:23:19.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:18 smithi049 ceph-mon[31942]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:23:20.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:20 smithi129 ceph-mon[36561]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:23:20.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:20 smithi154 ceph-mon[36471]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:23:21.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:20 smithi049 ceph-mon[31942]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:23:22.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:22 smithi129 ceph-mon[36561]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:22.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:22 smithi154 ceph-mon[36471]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:23.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:22 smithi049 ceph-mon[31942]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:23.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:23 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:23:23.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:23 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:23:24.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:23 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:23:24.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:24 smithi129 ceph-mon[36561]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:24.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:24 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:23:24.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:24 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:23:24.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:24 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:23:24.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:24 smithi154 ceph-mon[36471]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:24.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:24 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:23:24.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:24 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:23:24.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:24 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:23:25.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:24 smithi049 ceph-mon[31942]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:25.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:24 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:23:25.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:24 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:23:25.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:24 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:23:26.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:26 smithi129 ceph-mon[36561]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:26.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:26 smithi154 ceph-mon[36471]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:27.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:26 smithi049 ceph-mon[31942]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:28.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:28 smithi129 ceph-mon[36561]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:28.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:28 smithi154 ceph-mon[36471]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:28 smithi049 ceph-mon[31942]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:29.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:29 smithi154 ceph-mon[36471]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:30.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:29 smithi049 ceph-mon[31942]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:30.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:29 smithi129 ceph-mon[36561]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:32.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:31 smithi154 ceph-mon[36471]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:32.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:31 smithi049 ceph-mon[31942]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:32.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:31 smithi129 ceph-mon[36561]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:32.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:23:32.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 -- 172.21.15.49:0/789832091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 msgr2=0x7f9cf0106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 --2- 172.21.15.49:0/789832091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf0106a40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009930 tx=0x7f9cec02bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 -- 172.21.15.49:0/789832091 shutdown_connections 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 --2- 172.21.15.49:0/789832091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 --2- 172.21.15.49:0/789832091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 0x7f9cf0100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 --2- 172.21.15.49:0/789832091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf0106a40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 -- 172.21.15.49:0/789832091 >> 172.21.15.49:0/789832091 conn(0x7f9cf00777e0 msgr2=0x7f9cf0077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 -- 172.21.15.49:0/789832091 shutdown_connections 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.956+0000 7f9cf7475640 1 -- 172.21.15.49:0/789832091 wait complete. 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 Processor -- start 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 -- start start 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 0x7f9cf0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0111b70 con 0x7f9cf0106660 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01119f0 con 0x7f9cf0101010 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf7475640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0111cf0 con 0x7f9cf0100650 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56742/0 (socket says 172.21.15.49:56742) 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 -- 172.21.15.49:0/1717093372 learned_addr learned my addr 172.21.15.49:0/1717093372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5211640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 0x7f9cf0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 -- 172.21.15.49:0/1717093372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 msgr2=0x7f9cf0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 0x7f9cf0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 -- 172.21.15.49:0/1717093372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 msgr2=0x7f9cf01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf5a12640 1 -- 172.21.15.49:0/1717093372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf019d030 con 0x7f9cf0106660 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.957+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9cf5a12640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800eb10 tx=0x7f9ce800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9ce67fc640 1 -- 172.21.15.49:0/1717093372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8004b20 con 0x7f9cf0106660 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf019d320 con 0x7f9cf0106660 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9ce67fc640 1 -- 172.21.15.49:0/1717093372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce8004cc0 con 0x7f9cf0106660 2024-03-16T08:23:32.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9ce67fc640 1 -- 172.21.15.49:0/1717093372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8010760 con 0x7f9cf0106660 2024-03-16T08:23:32.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.958+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01a4bb0 con 0x7f9cf0106660 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9ce67fc640 1 -- 172.21.15.49:0/1717093372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9ce8005020 con 0x7f9cf0106660 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 Processor -- start 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 -- start start 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 0x562815227110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x562815229a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562815182500 con 0x562815227650 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56281512abe0 con 0x56281521e150 2024-03-16T08:23:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56281517e5a0 con 0x56281521ea70 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 0x562815227110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:43092/0 (socket says 172.21.15.49:43092) 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cf4a10640 1 -- 172.21.15.49:0/3729991002 learned_addr learned my addr 172.21.15.49:0/3729991002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x562815229a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.961+0000 7f9ce67fc640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 0x7f9cdc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5a12640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 0x7f9cdc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 -- 172.21.15.49:0/3729991002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 msgr2=0x56281521e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 -- 172.21.15.49:0/3729991002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 msgr2=0x562815229a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x562815229a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 -- 172.21.15.49:0/3729991002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281513e4b0 con 0x56281521ea70 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9ce67fc640 1 -- 172.21.15.49:0/1717093372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9ce80cf320 con 0x7f9cf0106660 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 0x562815227110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009230 tx=0x7f9cec009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6667e16ac2625992 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cf5a12640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 0x7f9cdc0a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80983a0 tx=0x7f9ce806d600 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cdb7fe640 1 -- 172.21.15.49:0/3729991002 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec042070 con 0x56281521ea70 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cdb7fe640 1 -- 172.21.15.49:0/3729991002 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cec02c690 con 0x56281521ea70 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.962+0000 7f9cdb7fe640 1 -- 172.21.15.49:0/3729991002 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004b90 con 0x56281521ea70 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3729991002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 msgr2=0x562815227110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 0x562815227110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009230 tx=0x7f9cec009610 comp rx=0 tx=0).stop 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3729991002 shutdown_connections 2024-03-16T08:23:32.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x562815229a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521ea70 0x562815227110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3729991002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x56281521e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3729991002 >> 172.21.15.49:0/3729991002 conn(0x562815218880 msgr2=0x562815218c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3729991002 shutdown_connections 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3729991002 wait complete. 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 -- start start 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.963+0000 7f9cfa05b740 1 Processor -- start 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x5628152246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562815227650 0x562815225140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56281522a3e0 con 0x56281521ea70 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56281513e160 con 0x56281521e150 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56281522a560 con 0x562815227650 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cfa05b740 10 client.0 _ll_register_callbacks cb 0x562815122150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56758/0 (socket says 172.21.15.49:56758) 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x5628152246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 -- 172.21.15.49:0/3257099174 learned_addr learned my addr 172.21.15.49:0/3257099174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562815227650 0x562815225140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 -- 172.21.15.49:0/3257099174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562815227650 msgr2=0x562815225140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562815227650 0x562815225140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 -- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 msgr2=0x5628152246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x5628152246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf5211640 1 -- 172.21.15.49:0/3257099174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562814f607d0 con 0x56281521ea70 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.964+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x5628152246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cf5211640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005ab0 tx=0x7f9cec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0488734b229838 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec042040 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9cec004040 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56281518fa40 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9cec004240 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 10 client.17049 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562814e963f0 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56281518fa40 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9cec003850 con 0x56281521ea70 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 10 client.17049 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 20 client.17049 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5628152374c0 0x5628152398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 4 client.17049 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.965+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562815189d90 con 0x5628152374c0 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.966+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5628152374c0 0x5628152398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.966+0000 7f9cd8ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.966+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5628152374c0 0x5628152398b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1da46d5fb5f0b2 server_cookie=a5c86d6bfbf09c76 in_seq=0 out_seq=0 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/3257099174 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562815189d90 con 0x5628152374c0 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cd8ff9640 10 client.17049 handle_command_reply: tid=0 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 client.17049 shutdown 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 msgr2=0x562815224c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005ab0 tx=0x7f9cec009680 comp rx=0 tx=0).stop 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 shutdown_connections 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5628152374c0 0x5628152398b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562815227650 0x562815225140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x562815224c00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cd8ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:32.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3257099174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521e150 0x5628152246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 >> 172.21.15.49:0/3257099174 conn(0x562815218880 msgr2=0x56281521a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 shutdown_connections 2024-03-16T08:23:32.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3257099174 wait complete. 2024-03-16T08:23:32.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.967+0000 7f9cfa05b740 20 client.17049 trim_cache size 0 max 16384 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.984+0000 7f9cfa05b740 1 Processor -- start 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 -- start start 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x562815226df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521ea70 0x562815218c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01afb90 con 0x562815227650 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01ad460 con 0x56281521ea70 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01ab290 con 0x56281521e150 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521ea70 0x562815218c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56774/0 (socket says 172.21.15.49:56774) 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 -- 172.21.15.49:0/3274119340 learned_addr learned my addr 172.21.15.49:0/3274119340 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x562815226df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 -- 172.21.15.49:0/3274119340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 msgr2=0x562815226df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x562815226df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 -- 172.21.15.49:0/3274119340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521ea70 msgr2=0x562815218c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521ea70 0x562815218c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 -- 172.21.15.49:0/3274119340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562814e560f0 con 0x562815227650 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.985+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0000c00 tx=0x7f9ce0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63b9b3badad5b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cdaffd640 1 -- 172.21.15.49:0/3274119340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004310 con 0x562815227650 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cdaffd640 1 -- 172.21.15.49:0/3274119340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce00044b0 con 0x562815227650 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cdaffd640 1 -- 172.21.15.49:0/3274119340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004db0 con 0x562815227650 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3274119340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 msgr2=0x56281523c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0000c00 tx=0x7f9ce0004730 comp rx=0 tx=0).stop 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3274119340 shutdown_connections 2024-03-16T08:23:32.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562815227650 0x56281523c8d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56281521ea70 0x562815218c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/3274119340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x562815226df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3274119340 >> 172.21.15.49:0/3274119340 conn(0x562815218880 msgr2=0x562815217070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3274119340 shutdown_connections 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.986+0000 7f9cfa05b740 1 -- 172.21.15.49:0/3274119340 wait complete. 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 -- start start 2024-03-16T08:23:32.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 Processor -- start 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x56281523c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 0x562815239a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01ab920 con 0x56281521ea70 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01acd90 con 0x562815227650 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01abff0 con 0x56281521e150 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cfa05b740 10 client.0 _ll_register_callbacks cb 0x7f9ce00116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56778/0 (socket says 172.21.15.49:56778) 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 -- 172.21.15.49:0/2731722918 learned_addr learned my addr 172.21.15.49:0/2731722918 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x56281523c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 0x562815239a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 -- 172.21.15.49:0/2731722918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 msgr2=0x56281523c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x56281523c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 -- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 msgr2=0x562815239a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 0x562815239a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf5211640 1 -- 172.21.15.49:0/2731722918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628151499d0 con 0x56281521ea70 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.987+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 0x562815239a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cf5211640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbe73e82e5ef1e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec05e060 con 0x56281521ea70 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f9cec0035f0 con 0x56281521ea70 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce00044b0 con 0x56281521ea70 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9cec042070 con 0x56281521ea70 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 10 client.17055 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:32.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56281529cd30 con 0x56281521ea70 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce00044b0 con 0x56281521ea70 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9cec059510 con 0x56281521ea70 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 10 client.17055 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 20 client.17055 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56281529d3c0 0x56281529f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 4 client.17055 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.988+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562815107540 con 0x56281529d3c0 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56281529d3c0 0x56281529f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cf4a10640 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56281529d3c0 0x56281529f7e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838f6aa96b0ba257 server_cookie=28c03a6ea5781c8a in_seq=0 out_seq=0 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cd8ff9640 10 client.17055 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cd8ff9640 1 -- 172.21.15.49:0/2731722918 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562815107540 con 0x56281529d3c0 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cd8ff9640 10 client.17055 handle_command_reply: tid=0 2024-03-16T08:23:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 client.17055 shutdown 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 msgr2=0x56281523c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec005b50 comp rx=0 tx=0).stop 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 shutdown_connections 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56281529d3c0 0x56281529f7e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562815227650 0x562815239a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cd8ff9640 0 client.17055 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56281521ea70 0x56281523c9d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 --2- 172.21.15.49:0/2731722918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56281521e150 0x56281523c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 >> 172.21.15.49:0/2731722918 conn(0x562815218880 msgr2=0x56281521a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.989+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 shutdown_connections 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cfa05b740 1 -- 172.21.15.49:0/2731722918 wait complete. 2024-03-16T08:23:32.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cfa05b740 20 client.17055 trim_cache size 0 max 16384 2024-03-16T08:23:32.997 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:32.997 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:23:32.997 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:23:32.997 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 msgr2=0x7f9cdc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 0x7f9cdc0a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80983a0 tx=0x7f9ce806d600 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 msgr2=0x7f9cf019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800eb10 tx=0x7f9ce800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 shutdown_connections 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9cdc0a2970 0x7f9cdc0a4e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cf0106660 0x7f9cf019c900 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cf0101010 0x7f9cf01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 --2- 172.21.15.49:0/1717093372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cf0100650 0x7f9cf0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 >> 172.21.15.49:0/1717093372 conn(0x7f9cf00777e0 msgr2=0x7f9cf0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 shutdown_connections 2024-03-16T08:23:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:32.990+0000 7f9cf7475640 1 -- 172.21.15.49:0/1717093372 wait complete. 2024-03-16T08:23:33.001 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:23:33.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:23:33.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.109+0000 7f061135b640 1 -- 172.21.15.49:0/1376682727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 msgr2=0x7f060c101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 --2- 172.21.15.49:0/1376682727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c101c00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009890 tx=0x7f05f402bb70 comp rx=0 tx=0).stop 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 -- 172.21.15.49:0/1376682727 shutdown_connections 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 --2- 172.21.15.49:0/1376682727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 --2- 172.21.15.49:0/1376682727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c101c00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 --2- 172.21.15.49:0/1376682727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 0x7f060c107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 -- 172.21.15.49:0/1376682727 >> 172.21.15.49:0/1376682727 conn(0x7f060c0fd270 msgr2=0x7f060c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 -- 172.21.15.49:0/1376682727 shutdown_connections 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 -- 172.21.15.49:0/1376682727 wait complete. 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 Processor -- start 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.110+0000 7f061135b640 1 -- start start 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 0x7f060c074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f060c111b60 con 0x7f060c102140 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1119e0 con 0x7f060c107790 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f060c111ce0 con 0x7f060c101780 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56804/0 (socket says 172.21.15.49:56804) 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 -- 172.21.15.49:0/2637874439 learned_addr learned my addr 172.21.15.49:0/2637874439 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060b7fe640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 0x7f060c074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060affd640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 -- 172.21.15.49:0/2637874439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 msgr2=0x7f060c073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 -- 172.21.15.49:0/2637874439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 msgr2=0x7f060c074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 0x7f060c074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 -- 172.21.15.49:0/2637874439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c1a6560 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f060a7fc640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f05f402c180 tx=0x7f05f4004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4041020 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c1a67f0 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f402c9f0 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4039040 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.111+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c1a6ce0 con 0x7f060c102140 2024-03-16T08:23:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.112+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c0685d0 con 0x7f060c102140 2024-03-16T08:23:33.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.113+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f05f4044020 con 0x7f060c102140 2024-03-16T08:23:33.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.114+0000 7f0603fff640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 0x7f05f80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.114+0000 7f060affd640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 0x7f05f80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.114+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f05f40f0550 con 0x7f060c102140 2024-03-16T08:23:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.115+0000 7f060affd640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 0x7f05f80a4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06040096c0 tx=0x7f0604009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.119+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05f404b020 con 0x7f060c102140 2024-03-16T08:23:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.344+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f060c109d10 con 0x7f060c102140 2024-03-16T08:23:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.345+0000 7f0603fff640 1 -- 172.21.15.49:0/2637874439 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f05f40bd410 con 0x7f060c102140 2024-03-16T08:23:33.345 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:23:33.345 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:23:33.346 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:23:33.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 msgr2=0x7f05f80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 0x7f05f80a4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06040096c0 tx=0x7f0604009260 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 msgr2=0x7f060c074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f05f402c180 tx=0x7f05f4004550 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 shutdown_connections 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f05f80a29a0 0x7f05f80a4e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c107790 0x7f060c074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f060c102140 0x7f060c074020 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 --2- 172.21.15.49:0/2637874439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c101780 0x7f060c073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 >> 172.21.15.49:0/2637874439 conn(0x7f060c0fd270 msgr2=0x7f060c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 shutdown_connections 2024-03-16T08:23:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.347+0000 7f061135b640 1 -- 172.21.15.49:0/2637874439 wait complete. 2024-03-16T08:23:33.360 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 -- 172.21.15.49:0/4060134732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 msgr2=0x7f591c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 --2- 172.21.15.49:0/4060134732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c102e30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02bcd0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 -- 172.21.15.49:0/4060134732 shutdown_connections 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 --2- 172.21.15.49:0/4060134732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c10be20 0x7f591c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 --2- 172.21.15.49:0/4060134732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c103370 0x7f591c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 --2- 172.21.15.49:0/4060134732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c102e30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 -- 172.21.15.49:0/4060134732 >> 172.21.15.49:0/4060134732 conn(0x7f591c077840 msgr2=0x7f591c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 -- 172.21.15.49:0/4060134732 shutdown_connections 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.470+0000 7f59218a1640 1 -- 172.21.15.49:0/4060134732 wait complete. 2024-03-16T08:23:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 Processor -- start 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 -- start start 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 0x7f591c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 0x7f591c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f591c111bf0 con 0x7f591c102a50 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f591c111a70 con 0x7f591c10be20 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f59218a1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f591c111d70 con 0x7f591c103370 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f591affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f591affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56822/0 (socket says 172.21.15.49:56822) 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.471+0000 7f591affd640 1 -- 172.21.15.49:0/131616949 learned_addr learned my addr 172.21.15.49:0/131616949 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591a7fc640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 0x7f591c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591b7fe640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 0x7f591c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 -- 172.21.15.49:0/131616949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 msgr2=0x7f591c100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 0x7f591c100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 -- 172.21.15.49:0/131616949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 msgr2=0x7f591c101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 0x7f591c101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 -- 172.21.15.49:0/131616949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1a6690 con 0x7f591c102a50 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591b7fe640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 0x7f591c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591a7fc640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 0x7f591c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f591affd640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f590c005940 tx=0x7f590c02cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f592089f640 1 -- 172.21.15.49:0/131616949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c041070 con 0x7f591c102a50 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f592089f640 1 -- 172.21.15.49:0/131616949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590c004450 con 0x7f591c102a50 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1a6920 con 0x7f591c102a50 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f592089f640 1 -- 172.21.15.49:0/131616949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c003d80 con 0x7f591c102a50 2024-03-16T08:23:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.472+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a6e10 con 0x7f591c102a50 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f592089f640 1 -- 172.21.15.49:0/131616949 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f590c044070 con 0x7f591c102a50 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 Processor -- start 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 -- start start 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483644c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x55848364d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5584835a8490 con 0x55848364dd30 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558483550c90 con 0x558483644830 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f5924569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584835a4530 con 0x558483645150 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.475+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56828/0 (socket says 172.21.15.49:56828) 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483644c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 -- 172.21.15.49:0/2786740944 learned_addr learned my addr 172.21.15.49:0/2786740944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591affd640 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x55848364d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f592089f640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 0x7f59040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591b7fe640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 0x7f59040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 -- 172.21.15.49:0/2786740944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 msgr2=0x55848364d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x55848364d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 -- 172.21.15.49:0/2786740944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 msgr2=0x558483644c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483644c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 -- 172.21.15.49:0/2786740944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55848360b500 con 0x55848364dd30 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f592089f640 1 -- 172.21.15.49:0/131616949 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f590c0f0960 con 0x7f591c102a50 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5910009cf0 tx=0x7f59100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca2cffc5781232c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f591b7fe640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 0x7f59040a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5914005fd0 tx=0x7f5914004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f5901ffb640 1 -- 172.21.15.49:0/2786740944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910012450 con 0x55848364dd30 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f5901ffb640 1 -- 172.21.15.49:0/2786740944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910012ad0 con 0x55848364dd30 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.476+0000 7f5901ffb640 1 -- 172.21.15.49:0/2786740944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591000fca0 con 0x55848364dd30 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- 172.21.15.49:0/2786740944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 msgr2=0x558483650120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5910009cf0 tx=0x7f59100117b0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- 172.21.15.49:0/2786740944 shutdown_connections 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483650120 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x55848364d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 --2- 172.21.15.49:0/2786740944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483644c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- 172.21.15.49:0/2786740944 >> 172.21.15.49:0/2786740944 conn(0x55848363ef60 msgr2=0x55848363f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- 172.21.15.49:0/2786740944 shutdown_connections 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- 172.21.15.49:0/2786740944 wait complete. 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.477+0000 7f5924569740 1 -- start start 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 Processor -- start 2024-03-16T08:23:33.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483644830 0x55848364ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 0x55848364b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558483650b20 con 0x558483645150 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558483564230 con 0x55848364dd30 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558483650ca0 con 0x558483644830 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f5924569740 10 client.0 _ll_register_callbacks cb 0x5584835a0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56842/0 (socket says 172.21.15.49:56842) 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 0x55848364b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 -- 172.21.15.49:0/2859544383 learned_addr learned my addr 172.21.15.49:0/2859544383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591affd640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483644830 0x55848364ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 -- 172.21.15.49:0/2859544383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483644830 msgr2=0x55848364ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483644830 0x55848364ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 -- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 msgr2=0x55848364b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 0x55848364b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591a7fc640 1 -- 172.21.15.49:0/2859544383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558483573100 con 0x558483645150 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.478+0000 7f591b7fe640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 0x55848364b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f591a7fc640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f591000a010 tx=0x7f5910005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5bd2bf4364513e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910029030 con 0x558483645150 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5910020040 con 0x558483645150 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5910012ad0 con 0x558483645150 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5910033050 con 0x558483645150 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 10 client.17073 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5584832bc3f0 con 0x558483645150 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910012ad0 con 0x558483645150 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f591000fa30 con 0x558483645150 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 10 client.17073 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 20 client.17073 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55848365e0b0 0x5584836604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 4 client.17073 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.479+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584835b59d0 con 0x55848365e0b0 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.480+0000 7f591affd640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55848365e0b0 0x5584836604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.480+0000 7f591affd640 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55848365e0b0 0x5584836604a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49d1b81427d2c419 server_cookie=35520f7b67a56631 in_seq=0 out_seq=0 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.480+0000 7f58faffd640 10 client.17073 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f58faffd640 1 -- 172.21.15.49:0/2859544383 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5584835b59d0 con 0x55848365e0b0 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f58faffd640 10 client.17073 handle_command_reply: tid=0 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 client.17073 shutdown 2024-03-16T08:23:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 msgr2=0x55848364b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f591000a010 tx=0x7f5910005cc0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 shutdown_connections 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55848365e0b0 0x5584836604a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55848364dd30 0x55848364b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x55848364b2a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f58faffd640 0 client.17073 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 --2- 172.21.15.49:0/2859544383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483644830 0x55848364ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 >> 172.21.15.49:0/2859544383 conn(0x55848363ef60 msgr2=0x558483640cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 shutdown_connections 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 1 -- 172.21.15.49:0/2859544383 wait complete. 2024-03-16T08:23:33.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.481+0000 7f5924569740 20 client.17073 trim_cache size 0 max 16384 2024-03-16T08:23:33.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 Processor -- start 2024-03-16T08:23:33.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 -- start start 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x55848364d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55848364dd30 0x558483649d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1b1f20 con 0x558483645150 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1af7f0 con 0x558483644830 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f5924569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ad620 con 0x55848364dd30 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56858/0 (socket says 172.21.15.49:56858) 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 -- 172.21.15.49:0/626333531 learned_addr learned my addr 172.21.15.49:0/626333531 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591affd640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x55848364d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591b7fe640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55848364dd30 0x558483649d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 -- 172.21.15.49:0/626333531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55848364dd30 msgr2=0x558483649d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55848364dd30 0x558483649d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 -- 172.21.15.49:0/626333531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 msgr2=0x55848364d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x55848364d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 -- 172.21.15.49:0/626333531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584833867d0 con 0x558483645150 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591affd640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x55848364d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f591a7fc640 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59100058e0 tx=0x7f59100099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f91b0f0adaae7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f59017fa640 1 -- 172.21.15.49:0/626333531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910029040 con 0x558483645150 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.495+0000 7f59017fa640 1 -- 172.21.15.49:0/626333531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591003c070 con 0x558483645150 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f59017fa640 1 -- 172.21.15.49:0/626333531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910012450 con 0x558483645150 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- 172.21.15.49:0/626333531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 msgr2=0x5584836598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59100058e0 tx=0x7f59100099f0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- 172.21.15.49:0/626333531 shutdown_connections 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55848364dd30 0x558483649d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558483645150 0x5584836598c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 --2- 172.21.15.49:0/626333531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x55848364d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- 172.21.15.49:0/626333531 >> 172.21.15.49:0/626333531 conn(0x55848363ef60 msgr2=0x558483649630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- 172.21.15.49:0/626333531 shutdown_connections 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- 172.21.15.49:0/626333531 wait complete. 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.496+0000 7f5924569740 1 -- start start 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 Processor -- start 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483650eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x5584836513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1adcb0 con 0x55848364dd30 2024-03-16T08:23:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1af120 con 0x558483644830 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ae380 con 0x558483645150 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f5924569740 10 client.0 _ll_register_callbacks cb 0x7f5914009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56862/0 (socket says 172.21.15.49:56862) 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 -- 172.21.15.49:0/389015139 learned_addr learned my addr 172.21.15.49:0/389015139 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591a7fc640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x5584836513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591affd640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483650eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 -- 172.21.15.49:0/389015139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 msgr2=0x5584836513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x5584836513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 -- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 msgr2=0x558483650eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483650eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 -- 172.21.15.49:0/389015139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584836c2450 con 0x55848364dd30 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f591b7fe640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f591400ba70 tx=0x7f5914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3461a1dd1a3fdd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.497+0000 7f58faffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914021bc0 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f5914021d60 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591003c070 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5914004b20 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 10 client.17079 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5584836c3790 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591003c070 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f591401e040 con 0x55848364dd30 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 10 client.17079 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 10 client.17079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 20 client.17079 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5584836c3f40 0x5584836c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 4 client.17079 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584835afd20 con 0x5584836c3f40 2024-03-16T08:23:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.498+0000 7f591affd640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5584836c3f40 0x5584836c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f58faffd640 10 client.17079 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f591affd640 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5584836c3f40 0x5584836c6360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b486171cccb99b8 server_cookie=3f40557734559f39 in_seq=0 out_seq=0 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f58faffd640 1 -- 172.21.15.49:0/389015139 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584835afd20 con 0x5584836c3f40 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f58faffd640 10 client.17079 handle_command_reply: tid=0 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 client.17079 shutdown 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 msgr2=0x558483651930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f591400ba70 tx=0x7f5914004a40 comp rx=0 tx=0).stop 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 shutdown_connections 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5584836c3f40 0x5584836c6360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55848364dd30 0x558483651930 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558483645150 0x5584836513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f58faffd640 0 client.17079 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 --2- 172.21.15.49:0/389015139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558483644830 0x558483650eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 >> 172.21.15.49:0/389015139 conn(0x55848363ef60 msgr2=0x558483649ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 shutdown_connections 2024-03-16T08:23:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 1 -- 172.21.15.49:0/389015139 wait complete. 2024-03-16T08:23:33.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.499+0000 7f5924569740 20 client.17079 trim_cache size 0 max 16384 2024-03-16T08:23:33.509 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:33.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 msgr2=0x7f59040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 0x7f59040a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5914005fd0 tx=0x7f5914004ab0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 msgr2=0x7f591c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f590c005940 tx=0x7f590c02cbd0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 shutdown_connections 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f59040a2970 0x7f59040a4e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10be20 0x7f591c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f591c103370 0x7f591c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 --2- 172.21.15.49:0/131616949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c102a50 0x7f591c1006a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 >> 172.21.15.49:0/131616949 conn(0x7f591c077840 msgr2=0x7f591c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 shutdown_connections 2024-03-16T08:23:33.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.500+0000 7f59218a1640 1 -- 172.21.15.49:0/131616949 wait complete. 2024-03-16T08:23:33.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 -- 172.21.15.49:0/2180006810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 msgr2=0x7f6450107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 --2- 172.21.15.49:0/2180006810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f6450107b40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6440009930 tx=0x7f644002bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 -- 172.21.15.49:0/2180006810 shutdown_connections 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 --2- 172.21.15.49:0/2180006810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450102110 0x7f645010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 --2- 172.21.15.49:0/2180006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450101750 0x7f6450101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 --2- 172.21.15.49:0/2180006810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f6450107b40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 -- 172.21.15.49:0/2180006810 >> 172.21.15.49:0/2180006810 conn(0x7f64500fd240 msgr2=0x7f64500ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 -- 172.21.15.49:0/2180006810 shutdown_connections 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.621+0000 7f6456a52640 1 -- 172.21.15.49:0/2180006810 wait complete. 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 Processor -- start 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 -- start start 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 0x7f6450198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450102110 0x7f6450198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111b30 con 0x7f6450107760 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64501119b0 con 0x7f6450101750 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6456a52640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111cb0 con 0x7f6450102110 2024-03-16T08:23:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.622+0000 7f6454fef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 0x7f6450198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56876/0 (socket says 172.21.15.49:56876) 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450102110 0x7f6450198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 -- 172.21.15.49:0/3845285547 learned_addr learned my addr 172.21.15.49:0/3845285547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 0x7f6450198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:43208/0 (socket says 172.21.15.49:43208) 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 -- 172.21.15.49:0/3845285547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450102110 msgr2=0x7f6450198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450102110 0x7f6450198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 -- 172.21.15.49:0/3845285547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 msgr2=0x7f6450198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 0x7f6450198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 -- 172.21.15.49:0/3845285547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019d110 con 0x7f6450107760 2024-03-16T08:23:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6454fef640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f644800efd0 tx=0x7f644800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644d7fa640 1 -- 172.21.15.49:0/3845285547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448003b70 con 0x7f6450107760 2024-03-16T08:23:33.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019d400 con 0x7f6450107760 2024-03-16T08:23:33.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644d7fa640 1 -- 172.21.15.49:0/3845285547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448003d10 con 0x7f6450107760 2024-03-16T08:23:33.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f644d7fa640 1 -- 172.21.15.49:0/3845285547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64480109f0 con 0x7f6450107760 2024-03-16T08:23:33.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.623+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a4c90 con 0x7f6450107760 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644d7fa640 1 -- 172.21.15.49:0/3845285547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6448004020 con 0x7f6450107760 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 Processor -- start 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 -- start start 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830213f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5583020a3490 con 0x55830213fbc0 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55830204bc90 con 0x55830213f210 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f645963f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55830209f530 con 0x55830214d140 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644d7fa640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 0x7f64380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830213f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56880/0 (socket says 172.21.15.49:56880) 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644ffff640 1 -- 172.21.15.49:0/2755591448 learned_addr learned my addr 172.21.15.49:0/2755591448 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644f7fe640 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f644d7fa640 1 -- 172.21.15.49:0/3845285547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f64480cf1d0 con 0x7f6450107760 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.626+0000 7f6454fef640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 0x7f64380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 -- 172.21.15.49:0/2755591448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 msgr2=0x55830214f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 -- 172.21.15.49:0/2755591448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 msgr2=0x55830214cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 -- 172.21.15.49:0/2755591448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558302106500 con 0x55830213f210 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644ffff640 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f644f7fe640 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830213f5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6444009730 tx=0x7f64440096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e127c6787013c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f6454fef640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 0x7f64380a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f644800cba0 tx=0x7f64480104c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f64367fc640 1 -- 172.21.15.49:0/2755591448 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444004050 con 0x55830213f210 2024-03-16T08:23:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f64367fc640 1 -- 172.21.15.49:0/2755591448 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6444005500 con 0x55830213f210 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.627+0000 7f64367fc640 1 -- 172.21.15.49:0/2755591448 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444004cc0 con 0x55830213f210 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- 172.21.15.49:0/2755591448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 msgr2=0x55830213f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830213f5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6444009730 tx=0x7f64440096c0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- 172.21.15.49:0/2755591448 shutdown_connections 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 --2- 172.21.15.49:0/2755591448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830213f5f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- 172.21.15.49:0/2755591448 >> 172.21.15.49:0/2755591448 conn(0x558302139f60 msgr2=0x55830213a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- 172.21.15.49:0/2755591448 shutdown_connections 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- 172.21.15.49:0/2755591448 wait complete. 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 -- start start 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.628+0000 7f645963f740 1 Processor -- start 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830214a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55830214ff00 con 0x55830213fbc0 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55830205f230 con 0x55830213f210 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558302150080 con 0x55830214d140 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 10 client.0 _ll_register_callbacks cb 0x55830209b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830214a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56896/0 (socket says 172.21.15.49:56896) 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 -- 172.21.15.49:0/1320017169 learned_addr learned my addr 172.21.15.49:0/1320017169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 -- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 msgr2=0x55830214abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 -- 172.21.15.49:0/1320017169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 msgr2=0x55830214a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830214a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 -- 172.21.15.49:0/1320017169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830206e100 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830214a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f644ffff640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6440009230 tx=0x7f6440009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9a32b32a84f4de server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f642f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440005d50 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f6440004240 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.629+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6444005500 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6440004440 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 10 client.17091 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558301db73f0 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6444005500 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f64400035f0 con 0x55830213fbc0 2024-03-16T08:23:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 10 client.17091 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 20 client.17091 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55830215d4c0 0x55830215f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 4 client.17091 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583020b09d0 con 0x55830215d4c0 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.630+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55830215d4c0 0x55830215f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.631+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55830215d4c0 0x55830215f910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f32d2d406941095 server_cookie=855fa585ff5bbf16 in_seq=0 out_seq=0 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.631+0000 7f642f7fe640 10 client.17091 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f642f7fe640 1 -- 172.21.15.49:0/1320017169 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583020b09d0 con 0x55830215d4c0 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f642f7fe640 10 client.17091 handle_command_reply: tid=0 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 client.17091 shutdown 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 msgr2=0x55830214a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6440009230 tx=0x7f6440009560 comp rx=0 tx=0).stop 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 shutdown_connections 2024-03-16T08:23:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55830215d4c0 0x55830215f910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830214abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f642f7fe640 0 client.17091 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x55830214a680 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 --2- 172.21.15.49:0/1320017169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x55830214a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 >> 172.21.15.49:0/1320017169 conn(0x558302139f60 msgr2=0x55830213bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 shutdown_connections 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 1 -- 172.21.15.49:0/1320017169 wait complete. 2024-03-16T08:23:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.632+0000 7f645963f740 20 client.17091 trim_cache size 0 max 16384 2024-03-16T08:23:33.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 Processor -- start 2024-03-16T08:23:33.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 -- start start 2024-03-16T08:23:33.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x558302158d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830213ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64501adb40 con 0x55830213fbc0 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64501ab410 con 0x55830213f210 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f645963f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64501a9260 con 0x55830214d140 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f644ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56900/0 (socket says 172.21.15.49:56900) 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f644ffff640 1 -- 172.21.15.49:0/1662173782 learned_addr learned my addr 172.21.15.49:0/1662173782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.645+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x558302158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830213ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 -- 172.21.15.49:0/1662173782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 msgr2=0x55830213ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830213ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 -- 172.21.15.49:0/1662173782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 msgr2=0x558302158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x558302158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 -- 172.21.15.49:0/1662173782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558301e817d0 con 0x55830213fbc0 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830213ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f644ffff640 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f644002cfd0 tx=0x7f644002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a492c4daad32593 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f6435ffb640 1 -- 172.21.15.49:0/1662173782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440054070 con 0x55830213fbc0 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f6435ffb640 1 -- 172.21.15.49:0/1662173782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644002cd40 con 0x55830213fbc0 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f6435ffb640 1 -- 172.21.15.49:0/1662173782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440004000 con 0x55830213fbc0 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f645963f740 1 -- 172.21.15.49:0/1662173782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 msgr2=0x5583021492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.646+0000 7f645963f740 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f644002cfd0 tx=0x7f644002bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- 172.21.15.49:0/1662173782 shutdown_connections 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830214d140 0x55830213ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830213fbc0 0x5583021492a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- 172.21.15.49:0/1662173782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x558302158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- 172.21.15.49:0/1662173782 >> 172.21.15.49:0/1662173782 conn(0x558302139f60 msgr2=0x55830213b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- 172.21.15.49:0/1662173782 shutdown_connections 2024-03-16T08:23:33.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- 172.21.15.49:0/1662173782 wait complete. 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- start start 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 Processor -- start 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x5583021552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830213fbc0 0x5583021506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64501a98d0 con 0x55830214d140 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64501aad40 con 0x55830213f210 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64501a9fa0 con 0x55830213fbc0 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f645963f740 10 client.0 _ll_register_callbacks cb 0x7f6440009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.647+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830213fbc0 0x5583021506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56904/0 (socket says 172.21.15.49:56904) 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 -- 172.21.15.49:0/1772599486 learned_addr learned my addr 172.21.15.49:0/1772599486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x5583021552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 -- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830213fbc0 msgr2=0x5583021506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830213fbc0 0x5583021506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 -- 172.21.15.49:0/1772599486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 msgr2=0x5583021552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x5583021552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 -- 172.21.15.49:0/1772599486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558302151170 con 0x55830214d140 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f644f7fe640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6444009dd0 tx=0x7f6444021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a09d1632d25e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f642f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444011050 con 0x55830214d140 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f64440111f0 con 0x55830214d140 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644002cd40 con 0x55830214d140 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f64440113f0 con 0x55830214d140 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f645963f740 10 client.17097 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5583021c3520 con 0x55830214d140 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.648+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644002cd40 con 0x55830214d140 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f644402f040 con 0x55830214d140 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 10 client.17097 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 10 client.17097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 20 client.17097 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5583021c39a0 0x5583021c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 4 client.17097 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583020aad20 con 0x5583021c39a0 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f644ffff640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5583021c39a0 0x5583021c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f644ffff640 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5583021c39a0 0x5583021c5d90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2403c3f9565fb937 server_cookie=1b5d528daa38e4c5 in_seq=0 out_seq=0 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f642f7fe640 10 client.17097 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f642f7fe640 1 -- 172.21.15.49:0/1772599486 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583020aad20 con 0x5583021c39a0 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f642f7fe640 10 client.17097 handle_command_reply: tid=0 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 1 client.17097 shutdown 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 msgr2=0x558302150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.649+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6444009dd0 tx=0x7f6444021040 comp rx=0 tx=0).stop 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 shutdown_connections 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5583021c39a0 0x5583021c5d90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55830214d140 0x558302150c30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55830213fbc0 0x5583021506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f642f7fe640 0 client.17097 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 --2- 172.21.15.49:0/1772599486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55830213f210 0x5583021552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 >> 172.21.15.49:0/1772599486 conn(0x558302139f60 msgr2=0x55830214a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 shutdown_connections 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 1 -- 172.21.15.49:0/1772599486 wait complete. 2024-03-16T08:23:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f645963f740 20 client.17097 trim_cache size 0 max 16384 2024-03-16T08:23:33.657 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 msgr2=0x7f64380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 0x7f64380a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f644800cba0 tx=0x7f64480104c0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 msgr2=0x7f645019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f644800efd0 tx=0x7f644800c6a0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 shutdown_connections 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f64380a2970 0x7f64380a4e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6450107760 0x7f645019c9e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6450102110 0x7f6450198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 --2- 172.21.15.49:0/3845285547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450101750 0x7f6450198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 >> 172.21.15.49:0/3845285547 conn(0x7f64500fd240 msgr2=0x7f64500fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 shutdown_connections 2024-03-16T08:23:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.650+0000 7f6456a52640 1 -- 172.21.15.49:0/3845285547 wait complete. 2024-03-16T08:23:33.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:23:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 -- 172.21.15.49:0/3766262976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38107790 msgr2=0x7f7c38107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/3766262976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38107790 0x7f7c38107b70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009930 tx=0x7f7c2802bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 -- 172.21.15.49:0/3766262976 shutdown_connections 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/3766262976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38102140 0x7f7c3810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/3766262976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 0x7f7c38101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/3766262976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38107790 0x7f7c38107b70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 -- 172.21.15.49:0/3766262976 >> 172.21.15.49:0/3766262976 conn(0x7f7c380fd210 msgr2=0x7f7c380ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 -- 172.21.15.49:0/3766262976 shutdown_connections 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.772+0000 7f7c3ccac640 1 -- 172.21.15.49:0/3766262976 wait complete. 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 Processor -- start 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 -- start start 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 0x7f7c381980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38107790 0x7f7c3819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111ba0 con 0x7f7c38102140 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111a20 con 0x7f7c38107790 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3ccac640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111d20 con 0x7f7c38101780 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c3659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 0x7f7c381980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56914/0 (socket says 172.21.15.49:56914) 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 -- 172.21.15.49:0/252597472 learned_addr learned my addr 172.21.15.49:0/252597472 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38107790 0x7f7c3819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 -- 172.21.15.49:0/252597472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 msgr2=0x7f7c381980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 0x7f7c381980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 -- 172.21.15.49:0/252597472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38107790 msgr2=0x7f7c3819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38107790 0x7f7c3819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.773+0000 7f7c35d9b640 1 -- 172.21.15.49:0/252597472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3819d090 con 0x7f7c38102140 2024-03-16T08:23:33.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ea60 tx=0x7f7c2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c277fe640 1 -- 172.21.15.49:0/252597472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00cde0 con 0x7f7c38102140 2024-03-16T08:23:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3819d380 con 0x7f7c38102140 2024-03-16T08:23:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c277fe640 1 -- 172.21.15.49:0/252597472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0035f0 con 0x7f7c38102140 2024-03-16T08:23:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c277fe640 1 -- 172.21.15.49:0/252597472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c010670 con 0x7f7c38102140 2024-03-16T08:23:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.774+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a4c10 con 0x7f7c38102140 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c277fe640 1 -- 172.21.15.49:0/252597472 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7c2c0037a0 con 0x7f7c38102140 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 Processor -- start 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 -- start start 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 0x56342c2a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56342c203490 con 0x56342c29f830 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56342c1abc90 con 0x56342c2a0150 2024-03-16T08:23:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3f972740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56342c1ff530 con 0x56342c2a8d30 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c36d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c36d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56922/0 (socket says 172.21.15.49:56922) 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4045893170 learned_addr learned my addr 172.21.15.49:0/4045893170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.777+0000 7f7c3659c640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 0x56342c2a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c277fe640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 0x7f7c200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4045893170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 msgr2=0x56342c2ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4045893170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 msgr2=0x56342c2a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 0x56342c2a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4045893170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342c266500 con 0x56342c29f830 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c3659c640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 0x56342c2a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c277fe640 1 -- 172.21.15.49:0/252597472 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7c2c0cf100 con 0x7f7c38102140 2024-03-16T08:23:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 0x7f7c200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c30011590 tx=0x7f7c30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a6913d7c792146 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c24ff9640 1 -- 172.21.15.49:0/4045893170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30017050 con 0x56342c29f830 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c24ff9640 1 -- 172.21.15.49:0/4045893170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3001bb90 con 0x56342c29f830 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.778+0000 7f7c24ff9640 1 -- 172.21.15.49:0/4045893170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000e3b0 con 0x56342c29f830 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 -- 172.21.15.49:0/4045893170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 msgr2=0x56342c29fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 0x7f7c200a4e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c30022000 tx=0x7f7c30023000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c30011590 tx=0x7f7c30011a50 comp rx=0 tx=0).stop 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 -- 172.21.15.49:0/4045893170 shutdown_connections 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a0150 0x56342c2a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4045893170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c29fc10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 -- 172.21.15.49:0/4045893170 >> 172.21.15.49:0/4045893170 conn(0x56342c299f60 msgr2=0x56342c29a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 -- 172.21.15.49:0/4045893170 shutdown_connections 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.779+0000 7f7c3f972740 1 -- 172.21.15.49:0/4045893170 wait complete. 2024-03-16T08:23:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 -- start start 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 Processor -- start 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a0150 0x56342c2a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56342c2abb20 con 0x56342c29f830 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56342c1bf230 con 0x56342c2a8d30 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56342c2abca0 con 0x56342c2a0150 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3f972740 10 client.0 _ll_register_callbacks cb 0x56342c1fb010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c36d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56932/0 (socket says 172.21.15.49:56932) 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c3659c640 1 -- 172.21.15.49:0/4284867871 learned_addr learned my addr 172.21.15.49:0/4284867871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.780+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a0150 0x56342c2a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4284867871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a0150 msgr2=0x56342c2a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a0150 0x56342c2a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4284867871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 msgr2=0x56342c2a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 -- 172.21.15.49:0/4284867871 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342c1ce100 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c3659c640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a67d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c3001d010 tx=0x7f7c30025880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02e4edc7fc15954 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000f780 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7c3001b990 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3001bb90 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c3000e3b0 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c3f972740 10 client.26990 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56342bf173f0 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3001bb90 con 0x56342c2a8d30 2024-03-16T08:23:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.781+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c3000ea90 con 0x56342c2a8d30 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 10 client.26990 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 20 client.26990 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c2b8fe0 0x56342c2bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 4 client.26990 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342c2109d0 con 0x56342c2b8fe0 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3659c640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c2b8fe0 0x56342c2bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c1dffb640 10 client.26990 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.782+0000 7f7c3659c640 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c2b8fe0 0x56342c2bb3d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2376119d2b25160e server_cookie=26b3983116cbbc29 in_seq=0 out_seq=0 2024-03-16T08:23:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c1dffb640 1 -- 172.21.15.49:0/4284867871 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56342c2109d0 con 0x56342c2b8fe0 2024-03-16T08:23:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c1dffb640 10 client.26990 handle_command_reply: tid=0 2024-03-16T08:23:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 client.26990 shutdown 2024-03-16T08:23:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 msgr2=0x56342c2a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a67d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c3001d010 tx=0x7f7c30025880 comp rx=0 tx=0).stop 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 shutdown_connections 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c2b8fe0 0x56342c2bb3d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a67d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a0150 0x56342c2a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 --2- 172.21.15.49:0/4284867871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c29f830 0x56342c2a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c1dffb640 0 client.26990 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.783+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 >> 172.21.15.49:0/4284867871 conn(0x56342c299f60 msgr2=0x56342c29bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.784+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 shutdown_connections 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.784+0000 7f7c3f972740 1 -- 172.21.15.49:0/4284867871 wait complete. 2024-03-16T08:23:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.784+0000 7f7c3f972740 20 client.26990 trim_cache size 0 max 16384 2024-03-16T08:23:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 Processor -- start 2024-03-16T08:23:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 -- start start 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c29f830 0x56342c2b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381afdd0 con 0x56342c2a0150 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ad6a0 con 0x56342c2a8d30 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3f972740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ab4d0 con 0x56342c29f830 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56942/0 (socket says 172.21.15.49:56942) 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 -- 172.21.15.49:0/2020489664 learned_addr learned my addr 172.21.15.49:0/2020489664 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c3659c640 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c29f830 0x56342c2b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c36d9d640 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 -- 172.21.15.49:0/2020489664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c29f830 msgr2=0x56342c2b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c29f830 0x56342c2b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 -- 172.21.15.49:0/2020489664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 msgr2=0x56342c2a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.797+0000 7f7c35d9b640 1 -- 172.21.15.49:0/2020489664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342bfe17d0 con 0x56342c2a0150 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d010 tx=0x7f7c2c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c5c2cb29d9eead server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c1ffff640 1 -- 172.21.15.49:0/2020489664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c09beb0 con 0x56342c2a0150 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c1ffff640 1 -- 172.21.15.49:0/2020489664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0a3f50 con 0x56342c2a0150 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c1ffff640 1 -- 172.21.15.49:0/2020489664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0ccb20 con 0x56342c2a0150 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 -- 172.21.15.49:0/2020489664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 msgr2=0x56342c2c0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d010 tx=0x7f7c2c086040 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 -- 172.21.15.49:0/2020489664 shutdown_connections 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c2a8d30 0x56342c2a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2c0a60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 --2- 172.21.15.49:0/2020489664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c29f830 0x56342c2b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.798+0000 7f7c3f972740 1 -- 172.21.15.49:0/2020489664 >> 172.21.15.49:0/2020489664 conn(0x56342c299f60 msgr2=0x56342c29bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- 172.21.15.49:0/2020489664 shutdown_connections 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- 172.21.15.49:0/2020489664 wait complete. 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- start start 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 Processor -- start 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 0x56342c2aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381abb60 con 0x56342c2a0150 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381acfd0 con 0x56342c29f830 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ac230 con 0x56342c2a8d30 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3f972740 10 client.0 _ll_register_callbacks cb 0x7f7c30011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c3659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 0x56342c2aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c35d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56952/0 (socket says 172.21.15.49:56952) 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c35d9b640 1 -- 172.21.15.49:0/53305910 learned_addr learned my addr 172.21.15.49:0/53305910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.799+0000 7f7c36d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 -- 172.21.15.49:0/53305910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 msgr2=0x56342c2ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 -- 172.21.15.49:0/53305910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 msgr2=0x56342c2acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 -- 172.21.15.49:0/53305910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342c31e2e0 con 0x56342c29f830 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3659c640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 0x56342c2aca20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802cfd0 tx=0x7f7c2802c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a33516cd580ad09 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280483b0 con 0x56342c29f830 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f7c28048550 con 0x56342c29f830 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2c0a3f50 con 0x56342c29f830 2024-03-16T08:23:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c28057050 con 0x56342c29f830 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3f972740 10 client.26996 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56342c31f620 con 0x56342c29f830 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.800+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0a3f50 con 0x56342c29f830 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c28004f50 con 0x56342c29f830 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 10 client.26996 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 20 client.26996 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c31fcf0 0x56342c3220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 4 client.26996 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56342c20ad20 con 0x56342c31fcf0 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c31fcf0 0x56342c3220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c35d9b640 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c31fcf0 0x56342c3220e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8da54e1089b7a8 server_cookie=72f787ade5fcbee in_seq=0 out_seq=0 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c1dffb640 10 client.26996 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c1dffb640 1 -- 172.21.15.49:0/53305910 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56342c20ad20 con 0x56342c31fcf0 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c1dffb640 10 client.26996 handle_command_reply: tid=0 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 client.26996 shutdown 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 msgr2=0x56342c2aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 0x56342c2aca20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802cfd0 tx=0x7f7c2802c3e0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 shutdown_connections 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56342c31fcf0 0x56342c3220e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56342c2a8d30 0x56342c2ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56342c2a0150 0x56342c2acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c3f972740 1 --2- 172.21.15.49:0/53305910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56342c29f830 0x56342c2aca20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.801+0000 7f7c1dffb640 0 client.26996 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 >> 172.21.15.49:0/53305910 conn(0x56342c299f60 msgr2=0x56342c2a62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 shutdown_connections 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3f972740 1 -- 172.21.15.49:0/53305910 wait complete. 2024-03-16T08:23:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3f972740 20 client.26996 trim_cache size 0 max 16384 2024-03-16T08:23:33.810 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 msgr2=0x7f7c200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 0x7f7c200a4e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c30022000 tx=0x7f7c30023000 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 msgr2=0x7f7c381985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ea60 tx=0x7f7c2c00ef20 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 shutdown_connections 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7c200a2970 0x7f7c200a4e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c38107790 0x7f7c3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c38102140 0x7f7c381985e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 --2- 172.21.15.49:0/252597472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c38101780 0x7f7c381980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 >> 172.21.15.49:0/252597472 conn(0x7f7c380fd210 msgr2=0x7f7c380fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 shutdown_connections 2024-03-16T08:23:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:33.802+0000 7f7c3ccac640 1 -- 172.21.15.49:0/252597472 wait complete. 2024-03-16T08:23:34.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:33 smithi049 ceph-mon[31942]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:34.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:33 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2637874439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:34.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:33 smithi154 ceph-mon[36471]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:34.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:33 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2637874439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:33 smithi129 ceph-mon[36561]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:33 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2637874439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:23:34.814 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:23:34.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:23:34.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:23:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 -- 172.21.15.49:0/2988964852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8324106710 msgr2=0x7f8324106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 --2- 172.21.15.49:0/2988964852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8324106710 0x7f8324106af0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8320009930 tx=0x7f832002bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 -- 172.21.15.49:0/2988964852 shutdown_connections 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 --2- 172.21.15.49:0/2988964852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83241010c0 0x7f832410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 --2- 172.21.15.49:0/2988964852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 --2- 172.21.15.49:0/2988964852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8324106710 0x7f8324106af0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.939+0000 7f832c11d640 1 -- 172.21.15.49:0/2988964852 >> 172.21.15.49:0/2988964852 conn(0x7f8324077840 msgr2=0x7f8324077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- 172.21.15.49:0/2988964852 shutdown_connections 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- 172.21.15.49:0/2988964852 wait complete. 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 Processor -- start 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- start start 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106710 0x7f832419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8324110ae0 con 0x7f83241010c0 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8324110960 con 0x7f8324100700 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.940+0000 7f832c11d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8324110c60 con 0x7f8324106710 2024-03-16T08:23:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:43298/0 (socket says 172.21.15.49:43298) 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 -- 172.21.15.49:0/4204399195 learned_addr learned my addr 172.21.15.49:0/4204399195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56974/0 (socket says 172.21.15.49:56974) 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f832a6ba640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106710 0x7f832419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 -- 172.21.15.49:0/4204399195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106710 msgr2=0x7f832419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106710 0x7f832419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 -- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 msgr2=0x7f83241986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 -- 172.21.15.49:0/4204399195 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832419d160 con 0x7f8324100700 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f83296b8640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f8329eb9640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8320009230 tx=0x7f832002cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f831affd640 1 -- 172.21.15.49:0/4204399195 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320041070 con 0x7f8324100700 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f831affd640 1 -- 172.21.15.49:0/4204399195 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320004450 con 0x7f8324100700 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832419d3f0 con 0x7f8324100700 2024-03-16T08:23:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f831affd640 1 -- 172.21.15.49:0/4204399195 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320003de0 con 0x7f8324100700 2024-03-16T08:23:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.941+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a4ce0 con 0x7f8324100700 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.943+0000 7f831affd640 1 -- 172.21.15.49:0/4204399195 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8320044070 con 0x7f8324100700 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f831affd640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 0x7f83100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 Processor -- start 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f831affd640 1 -- 172.21.15.49:0/4204399195 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f83200f0640 con 0x7f8324100700 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f83296b8640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 0x7f83100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 -- start start 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd84eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd90190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dce80d0 con 0x55fc5dd84a30 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.944+0000 7f832ede0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dc90c70 con 0x55fc5dd8dda0 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f832ede0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dce4170 con 0x55fc5dd85710 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd84eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:51950/0 (socket says 172.21.15.49:51950) 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 -- 172.21.15.49:0/1630678688 learned_addr learned my addr 172.21.15.49:0/1630678688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f83296b8640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 0x7f83100a4e30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8324078000 tx=0x7f8314005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f832a6ba640 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd90190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 -- 172.21.15.49:0/1630678688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 msgr2=0x55fc5dd90190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd90190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 -- 172.21.15.49:0/1630678688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 msgr2=0x55fc5dd84eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd84eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 -- 172.21.15.49:0/1630678688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5dac47e0 con 0x55fc5dd85710 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83200b9260 tx=0x7f8320009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a80ad0d81cc9a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f830ffff640 1 -- 172.21.15.49:0/1630678688 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832004b040 con 0x55fc5dd85710 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f830ffff640 1 -- 172.21.15.49:0/1630678688 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83200bce00 con 0x55fc5dd85710 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.945+0000 7f830ffff640 1 -- 172.21.15.49:0/1630678688 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200ee060 con 0x55fc5dd85710 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- 172.21.15.49:0/1630678688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 msgr2=0x55fc5dd84460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83200b9260 tx=0x7f8320009260 comp rx=0 tx=0).stop 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- 172.21.15.49:0/1630678688 shutdown_connections 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd90190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd84eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 --2- 172.21.15.49:0/1630678688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd84460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- 172.21.15.49:0/1630678688 >> 172.21.15.49:0/1630678688 conn(0x55fc5dd4bc50 msgr2=0x55fc5dd7f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- 172.21.15.49:0/1630678688 shutdown_connections 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- 172.21.15.49:0/1630678688 wait complete. 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.946+0000 7f832ede0740 1 -- start start 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 Processor -- start 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dd909d0 con 0x55fc5dd85710 2024-03-16T08:23:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dca40d0 con 0x55fc5dd84a30 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc5dd90b50 con 0x55fc5dd8dda0 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832ede0740 10 client.0 _ll_register_callbacks cb 0x55fc5dc6d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56996/0 (socket says 172.21.15.49:56996) 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f832a6ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 -- 172.21.15.49:0/1952453770 learned_addr learned my addr 172.21.15.49:0/1952453770 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 -- 172.21.15.49:0/1952453770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 msgr2=0x55fc5dd8b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd8b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 -- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 msgr2=0x55fc5dd8acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.947+0000 7f83296b8640 1 -- 172.21.15.49:0/1952453770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5db24f70 con 0x55fc5dd85710 2024-03-16T08:23:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f83296b8640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8314003740 tx=0x7f8314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6535ecb859ba453f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400f810 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f831400f9b0 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83200bce00 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f831400fbb0 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 10 client.17127 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc5d9bc170 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83200bce00 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f831401a070 con 0x55fc5dd85710 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 10 client.17127 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 20 client.17127 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5dd9dbb0 0x55fc5dda0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 4 client.17127 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.948+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5dcf5610 con 0x55fc5dd9dbb0 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.949+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5dd9dbb0 0x55fc5dda0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.949+0000 7f830d7fa640 10 client.17127 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.949+0000 7f8329eb9640 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5dd9dbb0 0x55fc5dda0030 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284ab32cdf487ea3 server_cookie=c18aae2b6939b071 in_seq=0 out_seq=0 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f830d7fa640 1 -- 172.21.15.49:0/1952453770 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fc5dcf5610 con 0x55fc5dd9dbb0 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f830d7fa640 10 client.17127 handle_command_reply: tid=0 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 client.17127 shutdown 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 msgr2=0x55fc5dd8b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8314003740 tx=0x7f8314004730 comp rx=0 tx=0).stop 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 shutdown_connections 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5dd9dbb0 0x55fc5dda0030 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd8b200 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 --2- 172.21.15.49:0/1952453770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f830d7fa640 0 client.17127 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 >> 172.21.15.49:0/1952453770 conn(0x55fc5dd4bc50 msgr2=0x55fc5dd811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 shutdown_connections 2024-03-16T08:23:34.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 1 -- 172.21.15.49:0/1952453770 wait complete. 2024-03-16T08:23:34.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.950+0000 7f832ede0740 20 client.17127 trim_cache size 0 max 16384 2024-03-16T08:23:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 Processor -- start 2024-03-16T08:23:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 -- start start 2024-03-16T08:23:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dda5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83241afcf0 con 0x55fc5dd84a30 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83241ad5c0 con 0x55fc5dd8dda0 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832ede0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83241ab3f0 con 0x55fc5dd85710 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57010/0 (socket says 172.21.15.49:57010) 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 -- 172.21.15.49:0/801618690 learned_addr learned my addr 172.21.15.49:0/801618690 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f83296b8640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f832a6ba640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dda5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 -- 172.21.15.49:0/801618690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 msgr2=0x55fc5dd99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 -- 172.21.15.49:0/801618690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 msgr2=0x55fc5dda5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dda5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f8329eb9640 1 -- 172.21.15.49:0/801618690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5dd83a60 con 0x55fc5dd84a30 2024-03-16T08:23:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.968+0000 7f83296b8640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f8329eb9640 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8320009430 tx=0x7f832002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eaaecf854b6cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f830f7fe640 1 -- 172.21.15.49:0/801618690 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832004b040 con 0x55fc5dd84a30 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f830f7fe640 1 -- 172.21.15.49:0/801618690 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83200bce00 con 0x55fc5dd84a30 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f830f7fe640 1 -- 172.21.15.49:0/801618690 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200b9470 con 0x55fc5dd84a30 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 -- 172.21.15.49:0/801618690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 msgr2=0x55fc5dd8d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8320009430 tx=0x7f832002bc30 comp rx=0 tx=0).stop 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 -- 172.21.15.49:0/801618690 shutdown_connections 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd8dda0 0x55fc5dda5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd85710 0x55fc5dd99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 --2- 172.21.15.49:0/801618690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd84a30 0x55fc5dd8d3f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 -- 172.21.15.49:0/801618690 >> 172.21.15.49:0/801618690 conn(0x55fc5dd4bc50 msgr2=0x55fc5dd81160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 -- 172.21.15.49:0/801618690 shutdown_connections 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.969+0000 7f832ede0740 1 -- 172.21.15.49:0/801618690 wait complete. 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 -- start start 2024-03-16T08:23:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 Processor -- start 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd95da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83241aba80 con 0x55fc5dd85710 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83241acef0 con 0x55fc5dd84a30 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83241ac150 con 0x55fc5dd8dda0 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832ede0740 10 client.0 _ll_register_callbacks cb 0x7f83140040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f8329eb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd95da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f83296b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57026/0 (socket says 172.21.15.49:57026) 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f83296b8640 1 -- 172.21.15.49:0/3773094099 learned_addr learned my addr 172.21.15.49:0/3773094099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.970+0000 7f832a6ba640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 -- 172.21.15.49:0/3773094099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 msgr2=0x55fc5dd932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 -- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 msgr2=0x55fc5dd95da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd95da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 -- 172.21.15.49:0/3773094099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5dd937f0 con 0x55fc5dd85710 2024-03-16T08:23:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f8329eb9640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd95da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f83296b8640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55fc5dd8a520 tx=0x7f8314003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ffa018b6d50ff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f830d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314034030 con 0x55fc5dd85710 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83200bce00 con 0x55fc5dd85710 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f8314023070 con 0x55fc5dd85710 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f831403d030 con 0x55fc5dd85710 2024-03-16T08:23:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f832ede0740 10 client.17133 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc5de04be0 con 0x55fc5dd85710 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.971+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83200bce00 con 0x55fc5dd85710 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f831401a040 con 0x55fc5dd85710 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 10 client.17133 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 20 client.17133 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5de05330 0x55fc5de07750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 4 client.17133 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc5dcef960 con 0x55fc5de05330 2024-03-16T08:23:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f8329eb9640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5de05330 0x55fc5de07750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f830d7fa640 10 client.17133 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.972+0000 7f8329eb9640 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5de05330 0x55fc5de07750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0722710cb11a32 server_cookie=5223a2f1fab04ace in_seq=0 out_seq=0 2024-03-16T08:23:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.973+0000 7f830d7fa640 1 -- 172.21.15.49:0/3773094099 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc5dcef960 con 0x55fc5de05330 2024-03-16T08:23:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.973+0000 7f830d7fa640 10 client.17133 handle_command_reply: tid=0 2024-03-16T08:23:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.973+0000 7f832ede0740 1 client.17133 shutdown 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.973+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 msgr2=0x55fc5dd92d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.973+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55fc5dd8a520 tx=0x7f8314003660 comp rx=0 tx=0).stop 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 shutdown_connections 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc5de05330 0x55fc5de07750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f830d7fa640 0 client.17133 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc5dd8dda0 0x55fc5dd932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc5dd85710 0x55fc5dd92d70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 --2- 172.21.15.49:0/3773094099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc5dd84a30 0x55fc5dd95da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 >> 172.21.15.49:0/3773094099 conn(0x55fc5dd4bc50 msgr2=0x55fc5dd7dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 shutdown_connections 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 1 -- 172.21.15.49:0/3773094099 wait complete. 2024-03-16T08:23:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832ede0740 20 client.17133 trim_cache size 0 max 16384 2024-03-16T08:23:34.982 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:34.982 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:23:34.982 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "1ba9e050-98d1-4509-ae54-561660817fe5", 2024-03-16T08:23:34.982 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:23:34.982 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 msgr2=0x7f83100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 0x7f83100a4e30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8324078000 tx=0x7f8314005f90 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 msgr2=0x7f8324198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8320009230 tx=0x7f832002cb30 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 shutdown_connections 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83100a2970 0x7f83100a4e30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106710 0x7f832419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83241010c0 0x7f83241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 --2- 172.21.15.49:0/4204399195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8324100700 0x7f8324198170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 >> 172.21.15.49:0/4204399195 conn(0x7f8324077840 msgr2=0x7f832406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 shutdown_connections 2024-03-16T08:23:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:34.974+0000 7f832c11d640 1 -- 172.21.15.49:0/4204399195 wait complete. 2024-03-16T08:23:34.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:23:34.986 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:23:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- 172.21.15.49:0/4151051586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8102140 msgr2=0x7f17b810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 --2- 172.21.15.49:0/4151051586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8102140 0x7f17b810e650 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009cd0 tx=0x7f17b002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- 172.21.15.49:0/4151051586 shutdown_connections 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 --2- 172.21.15.49:0/4151051586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8102140 0x7f17b810e650 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 --2- 172.21.15.49:0/4151051586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8101780 0x7f17b8101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 --2- 172.21.15.49:0/4151051586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8107790 0x7f17b8107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- 172.21.15.49:0/4151051586 >> 172.21.15.49:0/4151051586 conn(0x7f17b80fd210 msgr2=0x7f17b80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- 172.21.15.49:0/4151051586 shutdown_connections 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- 172.21.15.49:0/4151051586 wait complete. 2024-03-16T08:23:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 Processor -- start 2024-03-16T08:23:35.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.097+0000 7f17be7d4640 1 -- start start 2024-03-16T08:23:35.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8101780 0x7f17b8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8102140 0x7f17b8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111b60 con 0x7f17b8107790 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81119e0 con 0x7f17b8101780 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17be7d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111ce0 con 0x7f17b8102140 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57038/0 (socket says 172.21.15.49:57038) 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 -- 172.21.15.49:0/2287740943 learned_addr learned my addr 172.21.15.49:0/2287740943 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17b7fff640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8101780 0x7f17b8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17b77fe640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8102140 0x7f17b8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 -- 172.21.15.49:0/2287740943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8102140 msgr2=0x7f17b8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8102140 0x7f17b8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 -- 172.21.15.49:0/2287740943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8101780 msgr2=0x7f17b8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8101780 0x7f17b8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 -- 172.21.15.49:0/2287740943 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b819cfe0 con 0x7f17b8107790 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.098+0000 7f17bcd71640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f17b00348f0 tx=0x7f17b0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.099+0000 7f17b57fa640 1 -- 172.21.15.49:0/2287740943 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0041020 con 0x7f17b8107790 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.099+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b819d270 con 0x7f17b8107790 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.099+0000 7f17b57fa640 1 -- 172.21.15.49:0/2287740943 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b0032e80 con 0x7f17b8107790 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.099+0000 7f17b57fa640 1 -- 172.21.15.49:0/2287740943 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0039040 con 0x7f17b8107790 2024-03-16T08:23:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.099+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a4b60 con 0x7f17b8107790 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17b57fa640 1 -- 172.21.15.49:0/2287740943 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f17b0044020 con 0x7f17b8107790 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 Processor -- start 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 -- start start 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5566a9519500 con 0x5566a95b5a70 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5566a94c1be0 con 0x5566a95b5150 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.101+0000 7f17c13be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5566a95155a0 con 0x5566a95be650 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b57fa640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 0x7f17a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95be110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57042/0 (socket says 172.21.15.49:57042) 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b7fff640 1 -- 172.21.15.49:0/1997505301 learned_addr learned my addr 172.21.15.49:0/1997505301 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:43352/0 (socket says 172.21.15.49:43352) 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17bcd71640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 0x7f17a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 -- 172.21.15.49:0/1997505301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 msgr2=0x5566a95c0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95c0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 -- 172.21.15.49:0/1997505301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 msgr2=0x5566a95be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 -- 172.21.15.49:0/1997505301 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a94d54b0 con 0x5566a95b5150 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b57fa640 1 -- 172.21.15.49:0/2287740943 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f17b00f0930 con 0x7f17b8107790 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009730 tx=0x7f17ac0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1500bb65721c79b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f17bcd71640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 0x7f17a00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b9a80 tx=0x7f17b008cb80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f179e7fc640 1 -- 172.21.15.49:0/1997505301 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004050 con 0x5566a95b5150 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f179e7fc640 1 -- 172.21.15.49:0/1997505301 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac005500 con 0x5566a95b5150 2024-03-16T08:23:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.102+0000 7f179e7fc640 1 -- 172.21.15.49:0/1997505301 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004da0 con 0x5566a95b5150 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- 172.21.15.49:0/1997505301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 msgr2=0x5566a95b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009730 tx=0x7f17ac0096c0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- 172.21.15.49:0/1997505301 shutdown_connections 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 --2- 172.21.15.49:0/1997505301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5150 0x5566a95b5530 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- 172.21.15.49:0/1997505301 >> 172.21.15.49:0/1997505301 conn(0x5566a95af880 msgr2=0x5566a95afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- 172.21.15.49:0/1997505301 shutdown_connections 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- 172.21.15.49:0/1997505301 wait complete. 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.103+0000 7f17c13be740 1 -- start start 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 Processor -- start 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5150 0x5566a95bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5566a95c1440 con 0x5566a95b5a70 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5566a94d5160 con 0x5566a95be650 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5566a95c15c0 con 0x5566a95b5150 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17c13be740 10 client.0 _ll_register_callbacks cb 0x5566a94b9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57048/0 (socket says 172.21.15.49:57048) 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5150 0x5566a95bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 -- 172.21.15.49:0/2752163214 learned_addr learned my addr 172.21.15.49:0/2752163214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b77fe640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 -- 172.21.15.49:0/2752163214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5150 msgr2=0x5566a95bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5150 0x5566a95bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 -- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 msgr2=0x5566a95bc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 -- 172.21.15.49:0/2752163214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a92f77d0 con 0x5566a95b5a70 2024-03-16T08:23:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b77fe640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17b7fff640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009fd0 tx=0x7f17a8012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaf7eaf6c92449c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.104+0000 7f17977fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8004030 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f17a80041d0 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac005500 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f17a80043d0 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 10 client.17145 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566a922d3f0 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac005500 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f17a8003bf0 con 0x5566a95b5a70 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 10 client.17145 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 20 client.17145 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a95ce680 0x5566a95d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 4 client.17145 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a9526a40 con 0x5566a95ce680 2024-03-16T08:23:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.105+0000 7f17b77fe640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a95ce680 0x5566a95d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.106+0000 7f17b77fe640 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a95ce680 0x5566a95d0aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad13241d530647c0 server_cookie=cc2da8e2e2a25c8b in_seq=0 out_seq=0 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.106+0000 7f17977fe640 10 client.17145 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.106+0000 7f17977fe640 1 -- 172.21.15.49:0/2752163214 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566a9526a40 con 0x5566a95ce680 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.106+0000 7f17977fe640 10 client.17145 handle_command_reply: tid=0 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.106+0000 7f17c13be740 1 client.17145 shutdown 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 msgr2=0x5566a95bbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009fd0 tx=0x7f17a8012bd0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 shutdown_connections 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a95ce680 0x5566a95d0aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5a70 0x5566a95bbba0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17977fe640 0 client.17145 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 --2- 172.21.15.49:0/2752163214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5150 0x5566a95bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 >> 172.21.15.49:0/2752163214 conn(0x5566a95af880 msgr2=0x5566a95b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 shutdown_connections 2024-03-16T08:23:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 1 -- 172.21.15.49:0/2752163214 wait complete. 2024-03-16T08:23:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.107+0000 7f17c13be740 20 client.17145 trim_cache size 0 max 16384 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 Processor -- start 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 -- start start 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5a70 0x5566a95ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81afca0 con 0x5566a95b5150 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ad570 con 0x5566a95be650 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17c13be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ab3a0 con 0x5566a95b5a70 2024-03-16T08:23:35.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57060/0 (socket says 172.21.15.49:57060) 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.124+0000 7f17b77fe640 1 -- 172.21.15.49:0/1223501050 learned_addr learned my addr 172.21.15.49:0/1223501050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b7fff640 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5a70 0x5566a95ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 -- 172.21.15.49:0/1223501050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5a70 msgr2=0x5566a95ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5a70 0x5566a95ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 -- 172.21.15.49:0/1223501050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 msgr2=0x5566a95bbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 -- 172.21.15.49:0/1223501050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a91ed0f0 con 0x5566a95b5150 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009250 tx=0x7f17ac0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdff5034b1db7077 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f179dffb640 1 -- 172.21.15.49:0/1223501050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac02e380 con 0x5566a95b5150 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f179dffb640 1 -- 172.21.15.49:0/1223501050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac00fe90 con 0x5566a95b5150 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.125+0000 7f179dffb640 1 -- 172.21.15.49:0/1223501050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac01f7d0 con 0x5566a95b5150 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- 172.21.15.49:0/1223501050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 msgr2=0x5566a95ba5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009250 tx=0x7f17ac0096c0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- 172.21.15.49:0/1223501050 shutdown_connections 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95be650 0x5566a95bbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95b5a70 0x5566a95ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- 172.21.15.49:0/1223501050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95ba5a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- 172.21.15.49:0/1223501050 >> 172.21.15.49:0/1223501050 conn(0x5566a95af880 msgr2=0x5566a95b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- 172.21.15.49:0/1223501050 shutdown_connections 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- 172.21.15.49:0/1223501050 wait complete. 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- start start 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 Processor -- start 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5a70 0x5566a95b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81aba30 con 0x5566a95b5150 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81acea0 con 0x5566a95b5a70 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ac100 con 0x5566a95be650 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.126+0000 7f17c13be740 10 client.0 _ll_register_callbacks cb 0x7f17a8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:23:35.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5a70 0x5566a95b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57068/0 (socket says 172.21.15.49:57068) 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 -- 172.21.15.49:0/1245542273 learned_addr learned my addr 172.21.15.49:0/1245542273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 -- 172.21.15.49:0/1245542273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 msgr2=0x5566a95b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 -- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5a70 msgr2=0x5566a95b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5a70 0x5566a95b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 -- 172.21.15.49:0/1245542273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a94e09d0 con 0x5566a95b5150 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17b77fe640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x5566a95c71f0 tx=0x7f17ac009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b8581fa89e82f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17977fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac03f060 con 0x5566a95b5150 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f17ac02e380 con 0x5566a95b5150 2024-03-16T08:23:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac00fe90 con 0x5566a95b5150 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f17ac00fcf0 con 0x5566a95b5150 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17c13be740 10 client.17151 fetch_fsmap learned FSMap version 24 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.127+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566a9634cf0 con 0x5566a95b5150 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac00fe90 con 0x5566a95b5150 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f17ac02c030 con 0x5566a95b5150 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 10 client.17151 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 20 client.17151 populate_metadata read hostname 'smithi049' 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a9635260 0x5566a9637680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 4 client.17151 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566a9520d90 con 0x5566a9635260 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17b7fff640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a9635260 0x5566a9637680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17b7fff640 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a9635260 0x5566a9637680 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1022caf27823086 server_cookie=984f485198ef03d5 in_seq=0 out_seq=0 2024-03-16T08:23:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17977fe640 10 client.17151 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17977fe640 1 -- 172.21.15.49:0/1245542273 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566a9520d90 con 0x5566a9635260 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.128+0000 7f17977fe640 10 client.17151 handle_command_reply: tid=0 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 client.17151 shutdown 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 msgr2=0x5566a95b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x5566a95c71f0 tx=0x7f17ac009eb0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 shutdown_connections 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5566a9635260 0x5566a9637680 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5566a95be650 0x5566a95b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566a95b5a70 0x5566a95b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 --2- 172.21.15.49:0/1245542273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566a95b5150 0x5566a95b8410 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17977fe640 0 client.17151 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 >> 172.21.15.49:0/1245542273 conn(0x5566a95af880 msgr2=0x5566a95b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 shutdown_connections 2024-03-16T08:23:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 1 -- 172.21.15.49:0/1245542273 wait complete. 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17c13be740 20 client.17151 trim_cache size 0 max 16384 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "1ba9e050-98d1-4509-ae54-561660817fe5": { 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "1ba9e050-98d1-4509-ae54-561660817fe5", 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:23:35.136 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 msgr2=0x7f17a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 0x7f17a00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b9a80 tx=0x7f17b008cb80 comp rx=0 tx=0).stop 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 msgr2=0x7f17b819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f17b00348f0 tx=0x7f17b0032c90 comp rx=0 tx=0).stop 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 shutdown_connections 2024-03-16T08:23:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f17a00a2970 0x7f17a00a4e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b8107790 0x7f17b819c8b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.129+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b8102140 0x7f17b8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.130+0000 7f17be7d4640 1 --2- 172.21.15.49:0/2287740943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17b8101780 0x7f17b8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.130+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 >> 172.21.15.49:0/2287740943 conn(0x7f17b80fd210 msgr2=0x7f17b80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.130+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 shutdown_connections 2024-03-16T08:23:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:23:35.130+0000 7f17be7d4640 1 -- 172.21.15.49:0/2287740943 wait complete. 2024-03-16T08:23:35.140 INFO:tasks.cephfs.filesystem:scrub status for tag:1ba9e050-98d1-4509-ae54-561660817fe5 - {'path': '/', 'tag': '1ba9e050-98d1-4509-ae54-561660817fe5', 'options': 'recursive,force'} 2024-03-16T08:23:36.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:35 smithi154 ceph-mon[36471]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:36.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:35 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:23:36.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:35 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:36.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:35 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:23:36.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:35 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:35 smithi049 ceph-mon[31942]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:35 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:23:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:35 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:36.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:35 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:23:36.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:35 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:35 smithi129 ceph-mon[36561]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:35 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:23:36.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:35 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:36.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:35 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:23:36.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:35 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:23:38.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:37 smithi154 ceph-mon[36471]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:38.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:37 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:23:38.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:37 smithi049 ceph-mon[31942]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:38.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:37 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:23:38.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:37 smithi129 ceph-mon[36561]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:38.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:37 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:23:40.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:39 smithi154 ceph-mon[36471]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:40.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:39 smithi049 ceph-mon[31942]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:39 smithi129 ceph-mon[36561]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:42.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:41 smithi154 ceph-mon[36471]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:42.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:41 smithi049 ceph-mon[31942]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:41 smithi129 ceph-mon[36561]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:44.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:43 smithi154 ceph-mon[36471]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:44.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:43 smithi049 ceph-mon[31942]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:43 smithi129 ceph-mon[36561]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:46.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:45 smithi154 ceph-mon[36471]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:46.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:45 smithi049 ceph-mon[31942]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:45 smithi129 ceph-mon[36561]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:48.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:47 smithi154 ceph-mon[36471]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:48.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:47 smithi049 ceph-mon[31942]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:48.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:47 smithi129 ceph-mon[36561]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:23:50.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:49 smithi154 ceph-mon[36471]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:50.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:49 smithi049 ceph-mon[31942]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:49 smithi129 ceph-mon[36561]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:23:52.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:51 smithi154 ceph-mon[36471]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:52.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:51 smithi049 ceph-mon[31942]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:52.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:51 smithi129 ceph-mon[36561]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:54.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:53 smithi154 ceph-mon[36471]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:54.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:53 smithi049 ceph-mon[31942]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:53 smithi129 ceph-mon[36561]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:56.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:55 smithi154 ceph-mon[36471]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:56.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:55 smithi049 ceph-mon[31942]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:55 smithi129 ceph-mon[36561]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:23:58.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:57 smithi154 ceph-mon[36471]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:58.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:57 smithi049 ceph-mon[31942]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:23:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:57 smithi129 ceph-mon[36561]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:00.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:23:59 smithi154 ceph-mon[36471]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:00.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:23:59 smithi049 ceph-mon[31942]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:23:59 smithi129 ceph-mon[36561]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:02.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:01 smithi154 ceph-mon[36471]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:02.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:01 smithi049 ceph-mon[31942]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:01 smithi129 ceph-mon[36561]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:04.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:03 smithi154 ceph-mon[36471]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:04.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:03 smithi049 ceph-mon[31942]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:03 smithi129 ceph-mon[36561]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:05.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:24:05.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:24:05.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3264678200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 msgr2=0x7fb6d0101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3264678200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0101bf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009930 tx=0x7fb6c402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3264678200 shutdown_connections 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3264678200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 0x7fb6d010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3264678200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0101bf0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3264678200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 0x7fb6d0107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3264678200 >> 172.21.15.49:0/3264678200 conn(0x7fb6d0078be0 msgr2=0x7fb6d0078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3264678200 shutdown_connections 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.266+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3264678200 wait complete. 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 Processor -- start 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 -- start start 2024-03-16T08:24:05.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 0x7fb6d01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 0x7fb6d019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0111b90 con 0x7fb6d0101770 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0111a10 con 0x7fb6d0107780 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6d5e61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0111d10 con 0x7fb6d0102130 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49166/0 (socket says 172.21.15.49:49166) 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/3999564996 learned_addr learned my addr 172.21.15.49:0/3999564996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 0x7fb6d01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.267+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 0x7fb6d019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/3999564996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 msgr2=0x7fb6d01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 0x7fb6d01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/3999564996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 msgr2=0x7fb6d019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 0x7fb6d019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/3999564996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d019d060 con 0x7fb6d0101770 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e9f0 tx=0x7fb6c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6ccff9640 1 -- 172.21.15.49:0/3999564996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000cd80 con 0x7fb6d0101770 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6ccff9640 1 -- 172.21.15.49:0/3999564996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0010910 con 0x7fb6d0101770 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d019d350 con 0x7fb6d0101770 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6ccff9640 1 -- 172.21.15.49:0/3999564996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0018990 con 0x7fb6d0101770 2024-03-16T08:24:05.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.268+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01a4be0 con 0x7fb6d0101770 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.270+0000 7fb6ccff9640 1 -- 172.21.15.49:0/3999564996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fb6c0010ac0 con 0x7fb6d0101770 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 Processor -- start 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 -- start start 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573661530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c57366a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57366ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c5735c5500 con 0x55c57366a650 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c57356dbe0 con 0x55c573661a70 2024-03-16T08:24:05.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c5735c15a0 con 0x55c573661150 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6ccff9640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 0x7fb6b80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c57366a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57366ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573661530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57366ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49174/0 (socket says 172.21.15.49:49174) 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 0x7fb6b80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6ceffd640 1 -- 172.21.15.49:0/2826299030 learned_addr learned my addr 172.21.15.49:0/2826299030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 -- 172.21.15.49:0/2826299030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 msgr2=0x55c57366a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c57366a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 -- 172.21.15.49:0/2826299030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 msgr2=0x55c57366ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6ccff9640 1 -- 172.21.15.49:0/3999564996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb6c00cf0d0 con 0x7fb6d0101770 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57366ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.271+0000 7fb6cffff640 1 -- 172.21.15.49:0/2826299030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5735814b0 con 0x55c573661150 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.272+0000 7fb6cffff640 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573661530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005fd0 tx=0x7fb6c8011c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0be0f5b98eb19f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.272+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 0x7fb6b80a4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb6c801d000 tx=0x7fb6c801f000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.272+0000 7fb6b5ffb640 1 -- 172.21.15.49:0/2826299030 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8003970 con 0x55c573661150 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.272+0000 7fb6b5ffb640 1 -- 172.21.15.49:0/2826299030 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8022030 con 0x55c573661150 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.272+0000 7fb6b5ffb640 1 -- 172.21.15.49:0/2826299030 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80042b0 con 0x55c573661150 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2826299030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 msgr2=0x55c573661530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573661530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005fd0 tx=0x7fb6c8011c80 comp rx=0 tx=0).stop 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2826299030 shutdown_connections 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57366ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c57366a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2826299030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573661530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2826299030 >> 172.21.15.49:0/2826299030 conn(0x55c57365b880 msgr2=0x55c57365bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2826299030 shutdown_connections 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2826299030 wait complete. 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 -- start start 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.273+0000 7fb6d8a51740 1 Processor -- start 2024-03-16T08:24:05.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c573667bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c57366a650 0x55c573668130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c57366d3e0 con 0x55c573661150 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c573581160 con 0x55c573661a70 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c57366d560 con 0x55c57366a650 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 10 client.0 _ll_register_callbacks cb 0x55c573565150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49182/0 (socket says 172.21.15.49:49182) 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2395294704 learned_addr learned my addr 172.21.15.49:0/2395294704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c573667bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cffff640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c57366a650 0x55c573668130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2395294704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c57366a650 msgr2=0x55c573668130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c57366a650 0x55c573668130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 msgr2=0x55c573667bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c573667bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2395294704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5733a37d0 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c573667bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0097c90 tx=0x7fb6c009b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646f31756e751bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0026040 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb6c00a3a90 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.274+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c8022030 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb6c00cccf0 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 10 client.17163 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c5732d93f0 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8022030 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb6c00d1200 con 0x55c573661150 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 10 client.17163 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 20 client.17163 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c57367a6a0 0x55c57367ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 4 client.17163 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5735d2a40 con 0x55c57367a6a0 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.275+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c57367a6a0 0x55c57367ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.276+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c57367a6a0 0x55c57367ca90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7321f277a9b824 server_cookie=8261c5493839b362 in_seq=0 out_seq=0 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.276+0000 7fb6aeffd640 10 client.17163 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:05.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.276+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2395294704 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c5735d2a40 con 0x55c57367a6a0 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.276+0000 7fb6aeffd640 10 client.17163 handle_command_reply: tid=0 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.276+0000 7fb6d8a51740 1 client.17163 shutdown 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 msgr2=0x55c5736676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0097c90 tx=0x7fb6c009b250 comp rx=0 tx=0).stop 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 shutdown_connections 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c57367a6a0 0x55c57367ca90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c57366a650 0x55c573668130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c573667bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6aeffd640 0 client.17163 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2395294704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736676b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 >> 172.21.15.49:0/2395294704 conn(0x55c57365b880 msgr2=0x55c57365d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 shutdown_connections 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2395294704 wait complete. 2024-03-16T08:24:05.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.277+0000 7fb6d8a51740 20 client.17163 trim_cache size 0 max 16384 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.294+0000 7fb6d8a51740 1 Processor -- start 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.294+0000 7fb6d8a51740 1 -- start start 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573669de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c5736761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01afce0 con 0x55c57366a650 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01ad5b0 con 0x55c573661a70 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01ab3e0 con 0x55c573661150 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c5736761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49188/0 (socket says 172.21.15.49:49188) 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573669de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 -- 172.21.15.49:0/3521396211 learned_addr learned my addr 172.21.15.49:0/3521396211 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 -- 172.21.15.49:0/3521396211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 msgr2=0x55c573669de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573669de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 -- 172.21.15.49:0/3521396211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 msgr2=0x55c5736761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c5736761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 -- 172.21.15.49:0/3521396211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5736618a0 con 0x55c57366a650 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6cffff640 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb6c801eda0 tx=0x7fb6c80117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c8e69b6616cfe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6b57fa640 1 -- 172.21.15.49:0/3521396211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c801e850 con 0x55c57366a650 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6b57fa640 1 -- 172.21.15.49:0/3521396211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8003e50 con 0x55c57366a650 2024-03-16T08:24:05.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.295+0000 7fb6b57fa640 1 -- 172.21.15.49:0/3521396211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80058a0 con 0x55c57366a650 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- 172.21.15.49:0/3521396211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 msgr2=0x55c57365d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb6c801eda0 tx=0x7fb6c80117c0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- 172.21.15.49:0/3521396211 shutdown_connections 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c57366a650 0x55c57365d3c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c573661a70 0x55c5736761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/3521396211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661150 0x55c573669de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- 172.21.15.49:0/3521396211 >> 172.21.15.49:0/3521396211 conn(0x55c57365b880 msgr2=0x55c57365ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- 172.21.15.49:0/3521396211 shutdown_connections 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- 172.21.15.49:0/3521396211 wait complete. 2024-03-16T08:24:05.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.296+0000 7fb6d8a51740 1 -- start start 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 Processor -- start 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 0x55c5736649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c57366a650 0x55c573664f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01aba70 con 0x55c573661150 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01acee0 con 0x55c57366a650 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01ac140 con 0x55c573661a70 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6d8a51740 10 client.0 _ll_register_callbacks cb 0x7fb6c000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49202/0 (socket says 172.21.15.49:49202) 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2170138958 learned_addr learned my addr 172.21.15.49:0/2170138958 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 0x55c5736649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cffff640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c57366a650 0x55c573664f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2170138958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 msgr2=0x55c5736649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 0x55c5736649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c57366a650 msgr2=0x55c573664f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c57366a650 0x55c573664f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 -- 172.21.15.49:0/2170138958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5732990f0 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 0x55c5736649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6cf7fe640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb6c009b7c0 tx=0x7fb6c00a3d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc507fcca0ada69d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.297+0000 7fb6aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c009b600 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fb6c00cc560 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c8003e50 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb6c00b9050 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 10 client.17169 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c57358c9d0 con 0x55c573661150 2024-03-16T08:24:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8003e50 con 0x55c573661150 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb6c009ac80 con 0x55c573661150 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 10 client.17169 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 20 client.17169 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c5736e0ff0 0x55c5736e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 4 client.17169 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5735ccd90 con 0x55c5736e0ff0 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.298+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c5736e0ff0 0x55c5736e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6aeffd640 10 client.17169 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:05.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6ceffd640 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c5736e0ff0 0x55c5736e33e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933f382cea07a697 server_cookie=d1a38fa44bcc1a94 in_seq=0 out_seq=0 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6aeffd640 1 -- 172.21.15.49:0/2170138958 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5735ccd90 con 0x55c5736e0ff0 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6aeffd640 10 client.17169 handle_command_reply: tid=0 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 client.17169 shutdown 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 msgr2=0x55c5736644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb6c009b7c0 tx=0x7fb6c00a3d40 comp rx=0 tx=0).stop 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 shutdown_connections 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55c5736e0ff0 0x55c5736e33e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c57366a650 0x55c573664f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c573661a70 0x55c5736649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 --2- 172.21.15.49:0/2170138958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c573661150 0x55c5736644b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 >> 172.21.15.49:0/2170138958 conn(0x55c57365b880 msgr2=0x55c57365a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6aeffd640 0 client.17169 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 shutdown_connections 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 1 -- 172.21.15.49:0/2170138958 wait complete. 2024-03-16T08:24:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.299+0000 7fb6d8a51740 20 client.17169 trim_cache size 0 max 16384 2024-03-16T08:24:05.309 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:24:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:24:05.309 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 msgr2=0x7fb6b80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 0x7fb6b80a4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb6c801d000 tx=0x7fb6c801f000 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 msgr2=0x7fb6d0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e9f0 tx=0x7fb6c000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 shutdown_connections 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fb6b80a2970 0x7fb6b80a4e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0107780 0x7fb6d019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d0102130 0x7fb6d01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 --2- 172.21.15.49:0/3999564996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d0101770 0x7fb6d0198070 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 >> 172.21.15.49:0/3999564996 conn(0x7fb6d0078be0 msgr2=0x7fb6d0104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 shutdown_connections 2024-03-16T08:24:05.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.300+0000 7fb6d5e61640 1 -- 172.21.15.49:0/3999564996 wait complete. 2024-03-16T08:24:05.312 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:24:05.312 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- 172.21.15.49:0/2269964926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41066b0 msgr2=0x7f31e4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/2269964926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41066b0 0x7f31e4106a90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009930 tx=0x7f31e002bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- 172.21.15.49:0/2269964926 shutdown_connections 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/2269964926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e4101060 0x7f31e410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/2269964926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e41006a0 0x7f31e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/2269964926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41066b0 0x7f31e4106a90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- 172.21.15.49:0/2269964926 >> 172.21.15.49:0/2269964926 conn(0x7f31e4077780 msgr2=0x7f31e4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- 172.21.15.49:0/2269964926 shutdown_connections 2024-03-16T08:24:05.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- 172.21.15.49:0/2269964926 wait complete. 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 Processor -- start 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.420+0000 7f31ec7c6640 1 -- start start 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4101060 0x7f31e4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e41066b0 0x7f31e419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110a80 con 0x7f31e41006a0 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110900 con 0x7f31e41066b0 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110c00 con 0x7f31e4101060 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49232/0 (socket says 172.21.15.49:49232) 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 -- 172.21.15.49:0/1558217440 learned_addr learned my addr 172.21.15.49:0/1558217440 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ead63640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e41066b0 0x7f31e419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31e9d61640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4101060 0x7f31e4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 -- 172.21.15.49:0/1558217440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4101060 msgr2=0x7f31e4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4101060 0x7f31e4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 -- 172.21.15.49:0/1558217440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e41066b0 msgr2=0x7f31e419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e41066b0 0x7f31e419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 -- 172.21.15.49:0/1558217440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e419d090 con 0x7f31e41006a0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ea562640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009230 tx=0x7f31e002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0041070 con 0x7f31e41006a0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e0004450 con 0x7f31e41006a0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e419d320 con 0x7f31e41006a0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0003de0 con 0x7f31e41006a0 2024-03-16T08:24:05.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.421+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a4c10 con 0x7f31e41006a0 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.422+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4068a20 con 0x7f31e41006a0 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.424+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f31e0044070 con 0x7f31e41006a0 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.424+0000 7f31db7fe640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 0x7f31d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.424+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f31e00f0a10 con 0x7f31e41006a0 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.424+0000 7f31e9d61640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 0x7f31d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.425+0000 7f31e9d61640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 0x7f31d00a4e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31e406aef0 tx=0x7f31d4005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.428+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f31e00bd8d0 con 0x7f31e41006a0 2024-03-16T08:24:05.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.651+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e41a4de0 con 0x7f31e41006a0 2024-03-16T08:24:05.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.651+0000 7f31db7fe640 1 -- 172.21.15.49:0/1558217440 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f31e00bd1d0 con 0x7f31e41006a0 2024-03-16T08:24:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:24:05.653 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:24:05.653 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 msgr2=0x7f31d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 0x7f31d00a4e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31e406aef0 tx=0x7f31d4005f90 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 msgr2=0x7f31e41980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009230 tx=0x7f31e002cb30 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 shutdown_connections 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f31d00a2970 0x7f31d00a4e30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31e41066b0 0x7f31e419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4101060 0x7f31e4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 --2- 172.21.15.49:0/1558217440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31e41006a0 0x7f31e41980f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.653+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 >> 172.21.15.49:0/1558217440 conn(0x7f31e4077780 msgr2=0x7f31e4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.654+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 shutdown_connections 2024-03-16T08:24:05.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.654+0000 7f31ec7c6640 1 -- 172.21.15.49:0/1558217440 wait complete. 2024-03-16T08:24:05.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:24:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 -- 172.21.15.49:0/2617268068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b0106710 msgr2=0x7ff2b0106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/2617268068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b0106710 0x7ff2b0106af0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009930 tx=0x7ff2ac02bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 -- 172.21.15.49:0/2617268068 shutdown_connections 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/2617268068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b01010c0 0x7ff2b010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/2617268068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 0x7ff2b0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/2617268068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b0106710 0x7ff2b0106af0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 -- 172.21.15.49:0/2617268068 >> 172.21.15.49:0/2617268068 conn(0x7ff2b0077840 msgr2=0x7ff2b0077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 -- 172.21.15.49:0/2617268068 shutdown_connections 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.775+0000 7ff2b75b3640 1 -- 172.21.15.49:0/2617268068 wait complete. 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 Processor -- start 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 -- start start 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 0x7ff2b00fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 0x7ff2b00fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0110b20 con 0x7ff2b01010c0 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01109a0 con 0x7ff2b0100700 2024-03-16T08:24:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b75b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0110ca0 con 0x7ff2b0106710 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49240/0 (socket says 172.21.15.49:49240) 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/1604319668 learned_addr learned my addr 172.21.15.49:0/1604319668 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b534f640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 0x7ff2b00fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 0x7ff2b00fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/1604319668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 msgr2=0x7ff2b00fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 0x7ff2b00fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/1604319668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 msgr2=0x7ff2b00fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 0x7ff2b00fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/1604319668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01a6600 con 0x7ff2b01010c0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.776+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 0x7ff2b00fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e9f0 tx=0x7ff2a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2a27fc640 1 -- 172.21.15.49:0/1604319668 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400cd80 con 0x7ff2b01010c0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2a27fc640 1 -- 172.21.15.49:0/1604319668 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4010910 con 0x7ff2b01010c0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b01a68f0 con 0x7ff2b01010c0 2024-03-16T08:24:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2a27fc640 1 -- 172.21.15.49:0/1604319668 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4018990 con 0x7ff2b01010c0 2024-03-16T08:24:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.777+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01a6e40 con 0x7ff2b01010c0 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.779+0000 7ff2a27fc640 1 -- 172.21.15.49:0/1604319668 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ff2a4010ac0 con 0x7ff2b01010c0 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.779+0000 7ff2ba196740 1 Processor -- start 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 -- start start 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faaacc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 0x55d7faab8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d7faa10490 con 0x55d7faaad150 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa9b8c90 con 0x55d7faab5d30 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2ba196740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d7faa0c530 con 0x55d7faaac830 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b534f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 0x55d7faab8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2a27fc640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 0x7ff29c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b534f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49248/0 (socket says 172.21.15.49:49248) 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faaacc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b534f640 1 -- 172.21.15.49:0/3203596622 learned_addr learned my addr 172.21.15.49:0/3203596622 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 0x7ff29c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2a27fc640 1 -- 172.21.15.49:0/1604319668 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff2a40cf170 con 0x7ff2b01010c0 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3203596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 msgr2=0x55d7faaacc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faaacc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3203596622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 msgr2=0x55d7faab57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3203596622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7faa73500 con 0x55d7faab5d30 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faaacc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.780+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 0x55d7faab8120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cc910 tx=0x7ff2a4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9fa56897ce0e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 0x7ff29c0a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80094d0 tx=0x7ff2a8004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff29b7fe640 1 -- 172.21.15.49:0/3203596622 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40a3fc0 con 0x55d7faab5d30 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff29b7fe640 1 -- 172.21.15.49:0/3203596622 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4097f40 con 0x55d7faab5d30 2024-03-16T08:24:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff29b7fe640 1 -- 172.21.15.49:0/3203596622 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4018b30 con 0x55d7faab5d30 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 -- 172.21.15.49:0/3203596622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 msgr2=0x55d7faab8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 0x55d7faab8120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cc910 tx=0x7ff2a4018420 comp rx=0 tx=0).stop 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 -- 172.21.15.49:0/3203596622 shutdown_connections 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faab5d30 0x55d7faab8120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3203596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faaacc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.781+0000 7ff2ba196740 1 -- 172.21.15.49:0/3203596622 >> 172.21.15.49:0/3203596622 conn(0x55d7faaa6f60 msgr2=0x55d7faaa7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- 172.21.15.49:0/3203596622 shutdown_connections 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- 172.21.15.49:0/3203596622 wait complete. 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- start start 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 Processor -- start 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faab2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faab5d30 0x55d7faab3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d7faab8ac0 con 0x55d7faaad150 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa9cc230 con 0x55d7faaac830 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d7faab8c40 con 0x55d7faab5d30 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2b4b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2b534f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faab2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2ba196740 10 client.0 _ll_register_callbacks cb 0x55d7faa08010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2b4b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49262/0 (socket says 172.21.15.49:49262) 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3087161357 learned_addr learned my addr 172.21.15.49:0/3087161357 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.782+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faab5d30 0x55d7faab3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3087161357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faab5d30 msgr2=0x55d7faab3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faab5d30 0x55d7faab3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 msgr2=0x55d7faab2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faab2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 -- 172.21.15.49:0/3087161357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fa9db100 con 0x55d7faaad150 2024-03-16T08:24:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff2a409b5d0 tx=0x7ff2a409afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733fac5e659ded47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40b4060 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff2a4099bd0 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a4097f40 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2a40a3fc0 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2ba196740 10 client.17187 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d7fa7243f0 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4097f40 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.783+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff2a4018c50 con 0x55d7faaad150 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 10 client.17187 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 20 client.17187 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7faac5f40 0x55d7faac8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 4 client.17187 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7faa1d9d0 con 0x55d7faac5f40 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7faac5f40 0x55d7faac8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7faac5f40 0x55d7faac8330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=988367ddd4250828 server_cookie=8fe3ec9eb9f9e074 in_seq=0 out_seq=0 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.784+0000 7ff298ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff298ff9640 1 -- 172.21.15.49:0/3087161357 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d7faa1d9d0 con 0x55d7faac5f40 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff298ff9640 10 client.17187 handle_command_reply: tid=0 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 client.17187 shutdown 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 msgr2=0x55d7faab32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff2a409b5d0 tx=0x7ff2a409afb0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 shutdown_connections 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7faac5f40 0x55d7faac8330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faab5d30 0x55d7faab3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faaad150 0x55d7faab32d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff298ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3087161357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faab2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 >> 172.21.15.49:0/3087161357 conn(0x55d7faaa6f60 msgr2=0x55d7faaa8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 shutdown_connections 2024-03-16T08:24:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 1 -- 172.21.15.49:0/3087161357 wait complete. 2024-03-16T08:24:05.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.785+0000 7ff2ba196740 20 client.17187 trim_cache size 0 max 16384 2024-03-16T08:24:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.798+0000 7ff2ba196740 1 Processor -- start 2024-03-16T08:24:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.798+0000 7ff2ba196740 1 -- start start 2024-03-16T08:24:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faab54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaad150 0x55d7faac1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01b1fe0 con 0x55d7faab5d30 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01af8b0 con 0x55d7faaad150 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2ba196740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01ad6e0 con 0x55d7faaac830 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49276/0 (socket says 172.21.15.49:49276) 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 -- 172.21.15.49:0/2763427330 learned_addr learned my addr 172.21.15.49:0/2763427330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaad150 0x55d7faac1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b534f640 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faab54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 -- 172.21.15.49:0/2763427330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 msgr2=0x55d7faab54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faab54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 -- 172.21.15.49:0/2763427330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaad150 msgr2=0x55d7faac1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaad150 0x55d7faac1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 -- 172.21.15.49:0/2763427330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7faab1d40 con 0x55d7faab5d30 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800fe40 tx=0x7ff2a8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c9342919db6957 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff29affd640 1 -- 172.21.15.49:0/2763427330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8013190 con 0x55d7faab5d30 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff29affd640 1 -- 172.21.15.49:0/2763427330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8013330 con 0x55d7faab5d30 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.799+0000 7ff29affd640 1 -- 172.21.15.49:0/2763427330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8010430 con 0x55d7faab5d30 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- 172.21.15.49:0/2763427330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 msgr2=0x55d7faacb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800fe40 tx=0x7ff2a8009f90 comp rx=0 tx=0).stop 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- 172.21.15.49:0/2763427330 shutdown_connections 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faacb350 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaad150 0x55d7faac1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 --2- 172.21.15.49:0/2763427330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaac830 0x55d7faab54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- 172.21.15.49:0/2763427330 >> 172.21.15.49:0/2763427330 conn(0x55d7faaa6f60 msgr2=0x55d7faaa7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- 172.21.15.49:0/2763427330 shutdown_connections 2024-03-16T08:24:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- 172.21.15.49:0/2763427330 wait complete. 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 -- start start 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.800+0000 7ff2ba196740 1 Processor -- start 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faacafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaad150 0x55d7faacb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01add70 con 0x55d7faab5d30 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01af1e0 con 0x55d7faaac830 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01ae440 con 0x55d7faaad150 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2ba196740 10 client.0 _ll_register_callbacks cb 0x7ff2a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49292/0 (socket says 172.21.15.49:49292) 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 -- 172.21.15.49:0/3775218609 learned_addr learned my addr 172.21.15.49:0/3775218609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faacafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b4b4e640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaad150 0x55d7faacb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 -- 172.21.15.49:0/3775218609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaad150 msgr2=0x55d7faacb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaad150 0x55d7faacb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 -- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 msgr2=0x55d7faacafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faacafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 -- 172.21.15.49:0/3775218609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fa7ee7d0 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faacafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff2b5b50640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8004420 tx=0x7ff2a8004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa87c432e4a8a2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.801+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8013a10 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7ff2a8022480 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a8013330 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2a8022680 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 10 client.17193 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d7fab2c7d0 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8013330 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff2a801f040 con 0x55d7faab5d30 2024-03-16T08:24:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 10 client.17193 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 10 client.17193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 20 client.17193 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7fab2cde0 0x55d7fab2f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 4 client.17193 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7faa17d20 con 0x55d7fab2cde0 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.802+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7fab2cde0 0x55d7fab2f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2b534f640 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7fab2cde0 0x55d7fab2f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a511cc0ad291110b server_cookie=afb7d8f15f2b0bbe in_seq=0 out_seq=0 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff298ff9640 10 client.17193 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff298ff9640 1 -- 172.21.15.49:0/3775218609 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7faa17d20 con 0x55d7fab2cde0 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff298ff9640 10 client.17193 handle_command_reply: tid=0 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 client.17193 shutdown 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 msgr2=0x55d7faac9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8004420 tx=0x7ff2a8004780 comp rx=0 tx=0).stop 2024-03-16T08:24:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 shutdown_connections 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55d7fab2cde0 0x55d7fab2f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7faab5d30 0x55d7faac9460 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7faaad150 0x55d7faacb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 --2- 172.21.15.49:0/3775218609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7faaac830 0x55d7faacafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff298ff9640 0 client.17193 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 >> 172.21.15.49:0/3775218609 conn(0x55d7faaa6f60 msgr2=0x55d7faab35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 shutdown_connections 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 1 -- 172.21.15.49:0/3775218609 wait complete. 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.803+0000 7ff2ba196740 20 client.17193 trim_cache size 0 max 16384 2024-03-16T08:24:05.811 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 msgr2=0x7ff29c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 0x7ff29c0a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80094d0 tx=0x7ff2a8004ab0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 msgr2=0x7ff2b00fdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e9f0 tx=0x7ff2a400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 shutdown_connections 2024-03-16T08:24:05.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ff29c0a2970 0x7ff29c0a4e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2b0106710 0x7ff2b00fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2b01010c0 0x7ff2b00fdbc0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 --2- 172.21.15.49:0/1604319668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2b0100700 0x7ff2b00fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 >> 172.21.15.49:0/1604319668 conn(0x7ff2b0077840 msgr2=0x7ff2b006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 shutdown_connections 2024-03-16T08:24:05.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.804+0000 7ff2b75b3640 1 -- 172.21.15.49:0/1604319668 wait complete. 2024-03-16T08:24:05.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:24:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.924+0000 7fad0e47e640 1 -- 172.21.15.49:0/328010400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08100650 msgr2=0x7fad08100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.924+0000 7fad0e47e640 1 --2- 172.21.15.49:0/328010400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08100650 0x7fad08100ad0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7facfc009930 tx=0x7facfc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 -- 172.21.15.49:0/328010400 shutdown_connections 2024-03-16T08:24:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 --2- 172.21.15.49:0/328010400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08101010 0x7fad0810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 --2- 172.21.15.49:0/328010400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08100650 0x7fad08100ad0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 --2- 172.21.15.49:0/328010400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08106660 0x7fad08106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 -- 172.21.15.49:0/328010400 >> 172.21.15.49:0/328010400 conn(0x7fad080777e0 msgr2=0x7fad08077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 -- 172.21.15.49:0/328010400 shutdown_connections 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 -- 172.21.15.49:0/328010400 wait complete. 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 Processor -- start 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.925+0000 7fad0e47e640 1 -- start start 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 0x7fad08102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08106660 0x7fad08103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad08110a80 con 0x7fad08101010 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad08110900 con 0x7fad08100650 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad08110c00 con 0x7fad08106660 2024-03-16T08:24:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad07fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 0x7fad08102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49310/0 (socket says 172.21.15.49:49310) 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 -- 172.21.15.49:0/471837400 learned_addr learned my addr 172.21.15.49:0/471837400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad07fff640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 0x7fad08102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58280/0 (socket says 172.21.15.49:58280) 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08106660 0x7fad08103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 -- 172.21.15.49:0/471837400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08106660 msgr2=0x7fad08103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08106660 0x7fad08103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 -- 172.21.15.49:0/471837400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 msgr2=0x7fad08102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 0x7fad08102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 -- 172.21.15.49:0/471837400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad081a65f0 con 0x7fad08101010 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad077fe640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad057fa640 1 -- 172.21.15.49:0/471837400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc043020 con 0x7fad08101010 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad057fa640 1 -- 172.21.15.49:0/471837400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0054e0 con 0x7fad08101010 2024-03-16T08:24:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081a6880 con 0x7fad08101010 2024-03-16T08:24:05.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad057fa640 1 -- 172.21.15.49:0/471837400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc003b50 con 0x7fad08101010 2024-03-16T08:24:05.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.926+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a6dd0 con 0x7fad08101010 2024-03-16T08:24:05.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad057fa640 1 -- 172.21.15.49:0/471837400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7facfc046020 con 0x7fad08101010 2024-03-16T08:24:05.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 Processor -- start 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 -- start start 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53d150 0x55766c5457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c548120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55766c4a0490 con 0x55766c53c830 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55766c448c90 con 0x55766c545d30 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad11068740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55766c49c530 con 0x55766c53d150 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad0ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad0ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49324/0 (socket says 172.21.15.49:49324) 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad0ca1b640 1 -- 172.21.15.49:0/4107217008 learned_addr learned my addr 172.21.15.49:0/4107217008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad057fa640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 0x7facf00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad07fff640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53d150 0x55766c5457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.929+0000 7fad077fe640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c548120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 0x7facf00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad057fa640 1 -- 172.21.15.49:0/471837400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7facfc0f0d20 con 0x7fad08101010 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 -- 172.21.15.49:0/4107217008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53d150 msgr2=0x55766c5457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53d150 0x55766c5457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 -- 172.21.15.49:0/4107217008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 msgr2=0x55766c548120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c548120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 -- 172.21.15.49:0/4107217008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55766c503500 con 0x55766c53c830 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad077fe640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c548120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad00013b20 tx=0x7fad0000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb39ec8ca2f2f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7facee7fc640 1 -- 172.21.15.49:0/4107217008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00022020 con 0x55766c53c830 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7facee7fc640 1 -- 172.21.15.49:0/4107217008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00009bf0 con 0x55766c53c830 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7facee7fc640 1 -- 172.21.15.49:0/4107217008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00010a50 con 0x55766c53c830 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.930+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 0x7facf00a4e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad00023000 tx=0x7fad00021000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- 172.21.15.49:0/4107217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 msgr2=0x55766c53cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad00013b20 tx=0x7fad0000f3c0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- 172.21.15.49:0/4107217008 shutdown_connections 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c548120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53d150 0x55766c5457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 --2- 172.21.15.49:0/4107217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53c830 0x55766c53cc10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- 172.21.15.49:0/4107217008 >> 172.21.15.49:0/4107217008 conn(0x55766c536f60 msgr2=0x55766c537340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- 172.21.15.49:0/4107217008 shutdown_connections 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- 172.21.15.49:0/4107217008 wait complete. 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 -- start start 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.931+0000 7fad11068740 1 Processor -- start 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c542dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c543850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55766c548b20 con 0x55766c53d150 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55766c45c230 con 0x55766c545d30 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55766c548ca0 con 0x55766c53c830 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 10 client.0 _ll_register_callbacks cb 0x55766c498010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49334/0 (socket says 172.21.15.49:49334) 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 -- 172.21.15.49:0/3505421481 learned_addr learned my addr 172.21.15.49:0/3505421481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c543850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad07fff640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c542dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 -- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 msgr2=0x55766c542dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c542dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 -- 172.21.15.49:0/3505421481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 msgr2=0x55766c543850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c543850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 -- 172.21.15.49:0/3505421481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766c46b100 con 0x55766c53d150 2024-03-16T08:24:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad077fe640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba247bcdf9e8ffac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7face77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005690 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7facfc0ee160 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.932+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad00009bf0 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facfc0bdba0 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 10 client.17205 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55766c1b43f0 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00009bf0 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7facfc0ee700 con 0x55766c53d150 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 10 client.17205 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 20 client.17205 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c555fa0 0x55766c5583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 4 client.17205 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55766c4ad9d0 con 0x55766c555fa0 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.933+0000 7fad07fff640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c555fa0 0x55766c5583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad07fff640 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c555fa0 0x55766c5583b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5024624dc90b117 server_cookie=6e1d7b11ea818266 in_seq=0 out_seq=0 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7face77fe640 10 client.17205 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7face77fe640 1 -- 172.21.15.49:0/3505421481 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55766c4ad9d0 con 0x55766c555fa0 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7face77fe640 10 client.17205 handle_command_reply: tid=0 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 client.17205 shutdown 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 msgr2=0x55766c543310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc0035f0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 shutdown_connections 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c555fa0 0x55766c5583b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c543850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c543310 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 --2- 172.21.15.49:0/3505421481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c542dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7face77fe640 0 client.17205 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:05.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.934+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 >> 172.21.15.49:0/3505421481 conn(0x55766c536f60 msgr2=0x55766c538cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.935+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 shutdown_connections 2024-03-16T08:24:05.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.935+0000 7fad11068740 1 -- 172.21.15.49:0/3505421481 wait complete. 2024-03-16T08:24:05.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.935+0000 7fad11068740 20 client.17205 trim_cache size 0 max 16384 2024-03-16T08:24:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 Processor -- start 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 -- start start 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c5518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c53d150 0x55766c55dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad081b1ec0 con 0x55766c545d30 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad081af790 con 0x55766c53d150 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad11068740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad081ad5c0 con 0x55766c53c830 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad0ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad0ca1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49350/0 (socket says 172.21.15.49:49350) 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad0ca1b640 1 -- 172.21.15.49:0/577108322 learned_addr learned my addr 172.21.15.49:0/577108322 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad077fe640 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c53d150 0x55766c55dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.950+0000 7fad07fff640 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c5518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 -- 172.21.15.49:0/577108322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 msgr2=0x55766c5518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c5518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 -- 172.21.15.49:0/577108322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c53d150 msgr2=0x55766c55dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c53d150 0x55766c55dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 -- 172.21.15.49:0/577108322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55766c27e7d0 con 0x55766c545d30 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fad00009530 tx=0x7fad000105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fddd8401bf6f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7facedffb640 1 -- 172.21.15.49:0/577108322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00022030 con 0x55766c545d30 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7facedffb640 1 -- 172.21.15.49:0/577108322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00009720 con 0x55766c545d30 2024-03-16T08:24:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.951+0000 7facedffb640 1 -- 172.21.15.49:0/577108322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00013660 con 0x55766c545d30 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- 172.21.15.49:0/577108322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 msgr2=0x55766c54dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fad00009530 tx=0x7fad000105b0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- 172.21.15.49:0/577108322 shutdown_connections 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c545d30 0x55766c54dee0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c53d150 0x55766c55dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 --2- 172.21.15.49:0/577108322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c5518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- 172.21.15.49:0/577108322 >> 172.21.15.49:0/577108322 conn(0x55766c536f60 msgr2=0x55766c538240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- 172.21.15.49:0/577108322 shutdown_connections 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- 172.21.15.49:0/577108322 wait complete. 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 -- start start 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.952+0000 7fad11068740 1 Processor -- start 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c54dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c54a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad081adc50 con 0x55766c53d150 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad081af0c0 con 0x55766c545d30 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad081ae320 con 0x55766c53c830 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 10 client.0 _ll_register_callbacks cb 0x7facfc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49360/0 (socket says 172.21.15.49:49360) 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 -- 172.21.15.49:0/3883119391 learned_addr learned my addr 172.21.15.49:0/3883119391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:05.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad07fff640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c54dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad0ca1b640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c54a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 -- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 msgr2=0x55766c54dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c54dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 -- 172.21.15.49:0/3883119391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 msgr2=0x55766c54a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c54a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 -- 172.21.15.49:0/3883119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766c54acc0 con 0x55766c53d150 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad077fe640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7facfc02c820 tx=0x7facfc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515e103a277fb8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7face77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0eedd0 con 0x55766c53d150 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7facfc0c1140 con 0x55766c53d150 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.953+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad00009720 con 0x55766c53d150 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facfc0044d0 con 0x55766c53d150 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 10 client.17211 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55766c5bd030 con 0x55766c53d150 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00009720 con 0x55766c53d150 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7facfc0ee160 con 0x55766c53d150 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 10 client.17211 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 10 client.17211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 20 client.17211 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c5bd5e0 0x55766c5bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 4 client.17211 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55766c4a7d20 con 0x55766c5bd5e0 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.954+0000 7fad07fff640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c5bd5e0 0x55766c5bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7face77fe640 10 client.17211 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad07fff640 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c5bd5e0 0x55766c5bfa00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45bbb9e7b866818b server_cookie=648ab9858d0aa077 in_seq=0 out_seq=0 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7face77fe640 1 -- 172.21.15.49:0/3883119391 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55766c4a7d20 con 0x55766c5bd5e0 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7face77fe640 10 client.17211 handle_command_reply: tid=0 2024-03-16T08:24:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 client.17211 shutdown 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 msgr2=0x55766c54a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7facfc02c820 tx=0x7facfc009260 comp rx=0 tx=0).stop 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 shutdown_connections 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55766c5bd5e0 0x55766c5bfa00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55766c545d30 0x55766c54a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7face77fe640 0 client.17211 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55766c53d150 0x55766c54a240 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 --2- 172.21.15.49:0/3883119391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55766c53c830 0x55766c54dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 >> 172.21.15.49:0/3883119391 conn(0x55766c536f60 msgr2=0x55766c538240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 shutdown_connections 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 1 -- 172.21.15.49:0/3883119391 wait complete. 2024-03-16T08:24:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.955+0000 7fad11068740 20 client.17211 trim_cache size 0 max 16384 2024-03-16T08:24:05.963 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 msgr2=0x7facf00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 0x7facf00a4e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad00023000 tx=0x7fad00021000 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 msgr2=0x7fad08103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc004360 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 shutdown_connections 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7facf00a2970 0x7facf00a4e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad08106660 0x7fad08103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad08101010 0x7fad08103110 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 --2- 172.21.15.49:0/471837400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad08100650 0x7fad08102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 >> 172.21.15.49:0/471837400 conn(0x7fad080777e0 msgr2=0x7fad08068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 shutdown_connections 2024-03-16T08:24:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:05.956+0000 7fad0e47e640 1 -- 172.21.15.49:0/471837400 wait complete. 2024-03-16T08:24:05.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:24:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 -- 172.21.15.49:0/2495616943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 msgr2=0x7fa850106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 --2- 172.21.15.49:0/2495616943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa850106a50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009930 tx=0x7fa84c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 -- 172.21.15.49:0/2495616943 shutdown_connections 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 --2- 172.21.15.49:0/2495616943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa85010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 --2- 172.21.15.49:0/2495616943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 --2- 172.21.15.49:0/2495616943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa850106a50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 -- 172.21.15.49:0/2495616943 >> 172.21.15.49:0/2495616943 conn(0x7fa8500fc0e0 msgr2=0x7fa8500fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 -- 172.21.15.49:0/2495616943 shutdown_connections 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.076+0000 7fa85659d640 1 -- 172.21.15.49:0/2495616943 wait complete. 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 Processor -- start 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 -- start start 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa8501984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501109b0 con 0x7fa850106670 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa850110830 con 0x7fa850100f90 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85659d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa850110b30 con 0x7fa8501005d0 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa855d9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa855d9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49366/0 (socket says 172.21.15.49:49366) 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa855d9c640 1 -- 172.21.15.49:0/3568308 learned_addr learned my addr 172.21.15.49:0/3568308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa85559b640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.077+0000 7fa854d9a640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa8501984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 -- 172.21.15.49:0/3568308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 msgr2=0x7fa850197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 -- 172.21.15.49:0/3568308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 msgr2=0x7fa8501984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa8501984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 -- 172.21.15.49:0/3568308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85019cf50 con 0x7fa850106670 2024-03-16T08:24:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa85559b640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa854d9a640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa8501984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa855d9c640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa84800eb10 tx=0x7fa84800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa8467fc640 1 -- 172.21.15.49:0/3568308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004b20 con 0x7fa850106670 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85019d240 con 0x7fa850106670 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa8467fc640 1 -- 172.21.15.49:0/3568308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa848004cc0 con 0x7fa850106670 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa8467fc640 1 -- 172.21.15.49:0/3568308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848010760 con 0x7fa850106670 2024-03-16T08:24:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.078+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501a4ad0 con 0x7fa850106670 2024-03-16T08:24:06.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.080+0000 7fa8467fc640 1 -- 172.21.15.49:0/3568308 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa848005020 con 0x7fa850106670 2024-03-16T08:24:06.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 Processor -- start 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 -- start start 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b83e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b841310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555a2b799490 con 0x555a2b83ef20 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a2b741c90 con 0x555a2b835a20 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555a2b795530 con 0x555a2b836340 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa8467fc640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 0x7fa83c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58342/0 (socket says 172.21.15.49:58342) 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 -- 172.21.15.49:0/2907503418 learned_addr learned my addr 172.21.15.49:0/2907503418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa85559b640 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b83e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa855d9c640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 0x7fa83c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b841310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa8467fc640 1 -- 172.21.15.49:0/3568308 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa8480cf390 con 0x7fa850106670 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 -- 172.21.15.49:0/2907503418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 msgr2=0x555a2b83e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b83e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 -- 172.21.15.49:0/2907503418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 msgr2=0x555a2b841310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b841310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.081+0000 7fa854d9a640 1 -- 172.21.15.49:0/2907503418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a2b7fc500 con 0x555a2b835a20 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.082+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa840011be0 tx=0x7fa840009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc348e234bf00f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.082+0000 7fa855d9c640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 0x7fa83c0a4e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa84809b740 tx=0x7fa84800efd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.082+0000 7fa83b7fe640 1 -- 172.21.15.49:0/2907503418 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84000fc50 con 0x555a2b835a20 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.082+0000 7fa83b7fe640 1 -- 172.21.15.49:0/2907503418 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84000fdf0 con 0x555a2b835a20 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.082+0000 7fa83b7fe640 1 -- 172.21.15.49:0/2907503418 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84000e720 con 0x555a2b835a20 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2907503418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 msgr2=0x555a2b835e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa840011be0 tx=0x7fa840009ba0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2907503418 shutdown_connections 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b841310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b83e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2907503418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b835e00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2907503418 >> 172.21.15.49:0/2907503418 conn(0x555a2b830150 msgr2=0x555a2b830530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2907503418 shutdown_connections 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2907503418 wait complete. 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- start start 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 Processor -- start 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b836340 0x555a2b83c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.083+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555a2b841d10 con 0x555a2b83ef20 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a2b755230 con 0x555a2b836340 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555a2b841e90 con 0x555a2b835a20 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 10 client.0 _ll_register_callbacks cb 0x555a2b791010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49374/0 (socket says 172.21.15.49:49374) 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 -- 172.21.15.49:0/778625731 learned_addr learned my addr 172.21.15.49:0/778625731 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85559b640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b836340 0x555a2b83c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 -- 172.21.15.49:0/778625731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 msgr2=0x555a2b83bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 -- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b836340 msgr2=0x555a2b83c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b836340 0x555a2b83c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 -- 172.21.15.49:0/778625731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a2b764100 con 0x555a2b83ef20 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa854d9a640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa84000fdc0 tx=0x7fa84000e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae3345ea97a6fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84000e9d0 con 0x555a2b83ef20 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84000fdf0 con 0x555a2b83ef20 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa84000eb70 con 0x555a2b83ef20 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa84000ed70 con 0x555a2b83ef20 2024-03-16T08:24:06.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 10 client.17223 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.084+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a2b4ad3f0 con 0x555a2b83ef20 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84000fdf0 con 0x555a2b83ef20 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa84002c020 con 0x555a2b83ef20 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 10 client.17223 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 20 client.17223 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b84f140 0x555a2b851530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 4 client.17223 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a2b7a69d0 con 0x555a2b84f140 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85559b640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b84f140 0x555a2b851530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa838ff9640 10 client.17223 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.085+0000 7fa85559b640 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b84f140 0x555a2b851530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3f99baec0f8bd8 server_cookie=d5e2f008f471697c in_seq=0 out_seq=0 2024-03-16T08:24:06.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa838ff9640 1 -- 172.21.15.49:0/778625731 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555a2b7a69d0 con 0x555a2b84f140 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa838ff9640 10 client.17223 handle_command_reply: tid=0 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 client.17223 shutdown 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 msgr2=0x555a2b83ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa84000fdc0 tx=0x7fa84000e3e0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 shutdown_connections 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b84f140 0x555a2b851530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b83ca00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b836340 0x555a2b83c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa838ff9640 0 client.17223 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/778625731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 >> 172.21.15.49:0/778625731 conn(0x555a2b830150 msgr2=0x555a2b831ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 shutdown_connections 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 1 -- 172.21.15.49:0/778625731 wait complete. 2024-03-16T08:24:06.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.086+0000 7fa85a3ef740 20 client.17223 trim_cache size 0 max 16384 2024-03-16T08:24:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.100+0000 7fa85a3ef740 1 Processor -- start 2024-03-16T08:24:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.100+0000 7fa85a3ef740 1 -- start start 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b83ef20 0x555a2b856d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501afb30 con 0x555a2b836340 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ad400 con 0x555a2b83ef20 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ab230 con 0x555a2b835a20 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49382/0 (socket says 172.21.15.49:49382) 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 -- 172.21.15.49:0/2469893050 learned_addr learned my addr 172.21.15.49:0/2469893050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b83ef20 0x555a2b856d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 -- 172.21.15.49:0/2469893050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 msgr2=0x555a2b83e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 -- 172.21.15.49:0/2469893050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b83ef20 msgr2=0x555a2b856d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b83ef20 0x555a2b856d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 -- 172.21.15.49:0/2469893050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a2b5777d0 con 0x555a2b836340 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa854d9a640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa85559b640 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005ab0 tx=0x7fa84c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5118fc573a8fa39 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa83affd640 1 -- 172.21.15.49:0/2469893050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c049b10 con 0x555a2b836340 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa83affd640 1 -- 172.21.15.49:0/2469893050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c049cb0 con 0x555a2b836340 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.101+0000 7fa83affd640 1 -- 172.21.15.49:0/2469893050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c02cb00 con 0x555a2b836340 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2469893050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 msgr2=0x555a2b84aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005ab0 tx=0x7fa84c009260 comp rx=0 tx=0).stop 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2469893050 shutdown_connections 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b83ef20 0x555a2b856d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b836340 0x555a2b84aab0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/2469893050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b835a20 0x555a2b83e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2469893050 >> 172.21.15.49:0/2469893050 conn(0x555a2b830150 msgr2=0x555a2b831520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2469893050 shutdown_connections 2024-03-16T08:24:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- 172.21.15.49:0/2469893050 wait complete. 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 -- start start 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.102+0000 7fa85a3ef740 1 Processor -- start 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b8470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b8381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ab8c0 con 0x555a2b83ef20 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501acd30 con 0x555a2b835a20 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501abf90 con 0x555a2b836340 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85a3ef740 10 client.0 _ll_register_callbacks cb 0x7fa8400116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49386/0 (socket says 172.21.15.49:49386) 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 -- 172.21.15.49:0/1595232806 learned_addr learned my addr 172.21.15.49:0/1595232806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b8470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa85559b640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b8381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 -- 172.21.15.49:0/1595232806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 msgr2=0x555a2b8381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b8381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 -- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 msgr2=0x555a2b8470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b8470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 -- 172.21.15.49:0/1595232806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a2b8476d0 con 0x555a2b83ef20 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa854d9a640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa840004d40 tx=0x7fa84000e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95872bade46f0f38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84000fc20 con 0x555a2b83ef20 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.103+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7fa840004640 con 0x555a2b83ef20 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c049cb0 con 0x555a2b83ef20 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa840004840 con 0x555a2b83ef20 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 10 client.17229 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a2b8b4d40 con 0x555a2b83ef20 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c049cb0 con 0x555a2b83ef20 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa84002c040 con 0x555a2b83ef20 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 10 client.17229 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 20 client.17229 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b8b5490 0x555a2b8b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 4 client.17229 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a2b7a0d20 con 0x555a2b8b5490 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.104+0000 7fa85559b640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b8b5490 0x555a2b8b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa838ff9640 10 client.17229 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85559b640 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b8b5490 0x555a2b8b78b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9267b6d0e48a2d22 server_cookie=c2460329fbdebfe8 in_seq=0 out_seq=0 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa838ff9640 1 -- 172.21.15.49:0/1595232806 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a2b7a0d20 con 0x555a2b8b5490 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa838ff9640 10 client.17229 handle_command_reply: tid=0 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 client.17229 shutdown 2024-03-16T08:24:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 msgr2=0x555a2b838730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa840004d40 tx=0x7fa84000e3e0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 shutdown_connections 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555a2b8b5490 0x555a2b8b78b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a2b83ef20 0x555a2b838730 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa838ff9640 0 client.17229 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555a2b836340 0x555a2b8381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 --2- 172.21.15.49:0/1595232806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a2b835a20 0x555a2b8470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 >> 172.21.15.49:0/1595232806 conn(0x555a2b830150 msgr2=0x555a2b82f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 shutdown_connections 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 1 -- 172.21.15.49:0/1595232806 wait complete. 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.105+0000 7fa85a3ef740 20 client.17229 trim_cache size 0 max 16384 2024-03-16T08:24:06.113 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 msgr2=0x7fa83c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 0x7fa83c0a4e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa84809b740 tx=0x7fa84800efd0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 msgr2=0x7fa85019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa84800eb10 tx=0x7fa84800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 shutdown_connections 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa83c0a2970 0x7fa83c0a4e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa850106670 0x7fa85019c820 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa850100f90 0x7fa8501984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 --2- 172.21.15.49:0/3568308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8501005d0 0x7fa850197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 >> 172.21.15.49:0/3568308 conn(0x7fa8500fc0e0 msgr2=0x7fa8500fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 shutdown_connections 2024-03-16T08:24:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:06.106+0000 7fa85659d640 1 -- 172.21.15.49:0/3568308 wait complete. 2024-03-16T08:24:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:05 smithi154 ceph-mon[36471]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:24:06.225 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:05 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1558217440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:06.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:05 smithi049 ceph-mon[31942]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:24:06.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:05 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1558217440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:05 smithi129 ceph-mon[36561]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:24:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:05 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1558217440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:07.119 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:24:07.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:24:07.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:24:07.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- 172.21.15.49:0/4145173744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040689c0 msgr2=0x7f8404068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 --2- 172.21.15.49:0/4145173744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040689c0 0x7f8404068da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009900 tx=0x7f83f402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- 172.21.15.49:0/4145173744 shutdown_connections 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 --2- 172.21.15.49:0/4145173744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 0x7f8404110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 --2- 172.21.15.49:0/4145173744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040692e0 0x7f840410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 --2- 172.21.15.49:0/4145173744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040689c0 0x7f8404068da0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- 172.21.15.49:0/4145173744 >> 172.21.15.49:0/4145173744 conn(0x7f84040777e0 msgr2=0x7f8404077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- 172.21.15.49:0/4145173744 shutdown_connections 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- 172.21.15.49:0/4145173744 wait complete. 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 Processor -- start 2024-03-16T08:24:07.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.244+0000 7f840a5d4640 1 -- start start 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040689c0 0x7f840419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 0x7f840419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113d40 con 0x7f84040692e0 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113bc0 con 0x7f84040689c0 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f840a5d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113ec0 con 0x7f840410e180 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49402/0 (socket says 172.21.15.49:49402) 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 -- 172.21.15.49:0/2887655990 learned_addr learned my addr 172.21.15.49:0/2887655990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f8403fff640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040689c0 0x7f840419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f8408b71640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 0x7f840419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 -- 172.21.15.49:0/2887655990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 msgr2=0x7f840419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 0x7f840419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 -- 172.21.15.49:0/2887655990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040689c0 msgr2=0x7f840419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040689c0 0x7f840419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 -- 172.21.15.49:0/2887655990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840419f270 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.245+0000 7f84037fe640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e9f0 tx=0x7f83ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.246+0000 7f84017fa640 1 -- 172.21.15.49:0/2887655990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00cd80 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.246+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840419f560 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.246+0000 7f84017fa640 1 -- 172.21.15.49:0/2887655990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ec010910 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.246+0000 7f84017fa640 1 -- 172.21.15.49:0/2887655990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0188f0 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.246+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a6df0 con 0x7f84040692e0 2024-03-16T08:24:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f84017fa640 1 -- 172.21.15.49:0/2887655990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f83ec010ac0 con 0x7f84040692e0 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 Processor -- start 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 -- start start 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b92a30 0x565166b92eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b9e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565166af60d0 con 0x565166b9bda0 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.248+0000 7f840d1c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565166a9ec70 con 0x565166b92a30 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f840d1c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565166af2170 con 0x565166b93710 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f84017fa640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 0x7f83f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f84037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b9e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:46072/0 (socket says 172.21.15.49:46072) 2024-03-16T08:24:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8403fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b92a30 0x565166b92eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 -- 172.21.15.49:0/3498862697 learned_addr learned my addr 172.21.15.49:0/3498862697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 0x7f83f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f84017fa640 1 -- 172.21.15.49:0/2887655990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f83ec0cf200 con 0x7f84040692e0 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 -- 172.21.15.49:0/3498862697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b92a30 msgr2=0x565166b92eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b92a30 0x565166b92eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 -- 172.21.15.49:0/3498862697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 msgr2=0x565166b9e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b9e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 -- 172.21.15.49:0/3498862697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651668d27e0 con 0x565166b93710 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011d60 tx=0x7f83fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7e50468b9b1af5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f8408b71640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 0x7f83f00a4e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00fc90 tx=0x7f83fc018000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.249+0000 7f83fa7fc640 1 -- 172.21.15.49:0/3498862697 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011570 con 0x565166b93710 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f83fa7fc640 1 -- 172.21.15.49:0/3498862697 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc009d30 con 0x565166b93710 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f83fa7fc640 1 -- 172.21.15.49:0/3498862697 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc005820 con 0x565166b93710 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 -- 172.21.15.49:0/3498862697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 msgr2=0x565166b92460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011d60 tx=0x7f83fc00f790 comp rx=0 tx=0).stop 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 -- 172.21.15.49:0/3498862697 shutdown_connections 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b9e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b92a30 0x565166b92eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3498862697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b93710 0x565166b92460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 -- 172.21.15.49:0/3498862697 >> 172.21.15.49:0/3498862697 conn(0x565166b59c50 msgr2=0x565166b8d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 -- 172.21.15.49:0/3498862697 shutdown_connections 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.250+0000 7f840d1c4740 1 -- 172.21.15.49:0/3498862697 wait complete. 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 -- start start 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 Processor -- start 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166b98dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565166b9eb50 con 0x565166b9bda0 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565166ab20d0 con 0x565166b93710 2024-03-16T08:24:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565166b9ecd0 con 0x565166b92a30 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f840d1c4740 10 client.0 _ll_register_callbacks cb 0x565166a7b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f8408b71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f8403fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166b98dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f8408b71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49406/0 (socket says 172.21.15.49:49406) 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f8408b71640 1 -- 172.21.15.49:0/3523818597 learned_addr learned my addr 172.21.15.49:0/3523818597 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.251+0000 7f84037fe640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 -- 172.21.15.49:0/3523818597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 msgr2=0x565166b98dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166b98dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 -- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 msgr2=0x565166b99310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 -- 172.21.15.49:0/3523818597 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565166932f70 con 0x565166b9bda0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f8408b71640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ee0 tx=0x7f83fc011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb07efff2036cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc033070 con 0x565166b9bda0 2024-03-16T08:24:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f83fc004a00 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc009d30 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f83fc03f050 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f840d1c4740 10 client.17241 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651667ca170 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc009d30 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.252+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f83fc024030 con 0x565166b9bda0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 10 client.17241 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 20 client.17241 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166babdc0 0x565166bae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 4 client.17241 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565166b03610 con 0x565166babdc0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f8403fff640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166babdc0 0x565166bae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f8403fff640 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166babdc0 0x565166bae1b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc4da9f0cc8fb4f server_cookie=f353dddac0c8fe09 in_seq=0 out_seq=0 2024-03-16T08:24:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.253+0000 7f83e37fe640 10 client.17241 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f83e37fe640 1 -- 172.21.15.49:0/3523818597 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565166b03610 con 0x565166babdc0 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f83e37fe640 10 client.17241 handle_command_reply: tid=0 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 client.17241 shutdown 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 msgr2=0x565166b99850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ee0 tx=0x7f83fc011600 comp rx=0 tx=0).stop 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 shutdown_connections 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166babdc0 0x565166bae1b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b9bda0 0x565166b99850 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f83e37fe640 0 client.17241 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 --2- 172.21.15.49:0/3523818597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166b98dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 >> 172.21.15.49:0/3523818597 conn(0x565166b59c50 msgr2=0x565166b8f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 shutdown_connections 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 1 -- 172.21.15.49:0/3523818597 wait complete. 2024-03-16T08:24:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.254+0000 7f840d1c4740 20 client.17241 trim_cache size 0 max 16384 2024-03-16T08:24:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 Processor -- start 2024-03-16T08:24:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 -- start start 2024-03-16T08:24:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 0x565166bb3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b1e20 con 0x565166b92a30 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84041af6f0 con 0x565166b93710 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f840d1c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84041ad520 con 0x565166b9bda0 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49420/0 (socket says 172.21.15.49:49420) 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 -- 172.21.15.49:0/2787846886 learned_addr learned my addr 172.21.15.49:0/2787846886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f84037fe640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8408b71640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 0x565166bb3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 -- 172.21.15.49:0/2787846886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 msgr2=0x565166bb3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 0x565166bb3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 -- 172.21.15.49:0/2787846886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 msgr2=0x565166b99670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 -- 172.21.15.49:0/2787846886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565166b9e860 con 0x565166b92a30 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8408b71640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 0x565166bb3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.272+0000 7f8403fff640 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f83f40096c0 tx=0x7f83f402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41ba4ef1567b634 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f83f9ffb640 1 -- 172.21.15.49:0/2787846886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f404d070 con 0x565166b92a30 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f83f9ffb640 1 -- 172.21.15.49:0/2787846886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4049dd0 con 0x565166b92a30 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f83f9ffb640 1 -- 172.21.15.49:0/2787846886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f402cbe0 con 0x565166b92a30 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 -- 172.21.15.49:0/2787846886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 msgr2=0x565166b9b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f83f40096c0 tx=0x7f83f402e040 comp rx=0 tx=0).stop 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 -- 172.21.15.49:0/2787846886 shutdown_connections 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b9bda0 0x565166bb3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b93710 0x565166b99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2787846886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b92a30 0x565166b9b500 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 -- 172.21.15.49:0/2787846886 >> 172.21.15.49:0/2787846886 conn(0x565166b59c50 msgr2=0x565166b8bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 -- 172.21.15.49:0/2787846886 shutdown_connections 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.273+0000 7f840d1c4740 1 -- 172.21.15.49:0/2787846886 wait complete. 2024-03-16T08:24:07.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 -- start start 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 Processor -- start 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166ba41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b9bda0 0x565166b955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84041adbb0 con 0x565166b93710 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84041af020 con 0x565166b9bda0 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84041ae280 con 0x565166b92a30 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f840d1c4740 10 client.0 _ll_register_callbacks cb 0x7f83ec097c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f84037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f84037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49436/0 (socket says 172.21.15.49:49436) 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f84037fe640 1 -- 172.21.15.49:0/2180210072 learned_addr learned my addr 172.21.15.49:0/2180210072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f8408b71640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b9bda0 0x565166b955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.274+0000 7f8403fff640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166ba41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 -- 172.21.15.49:0/2180210072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 msgr2=0x565166ba41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166ba41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 -- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b9bda0 msgr2=0x565166b955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b9bda0 0x565166b955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 -- 172.21.15.49:0/2180210072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565166b95df0 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f84037fe640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005b60 tx=0x7f83ec06d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fef5b1bd6e8bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec084e60 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f83ec0a3d80 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f4049dd0 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f83ec026040 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 10 client.17247 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565166c13030 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4049dd0 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f83ec06d0f0 con 0x565166b93710 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 10 client.17247 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:07.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 20 client.17247 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166c13800 0x565166c15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 4 client.17247 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.275+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565166afd960 con 0x565166c13800 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.276+0000 7f8403fff640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166c13800 0x565166c15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.276+0000 7f83e37fe640 10 client.17247 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.276+0000 7f8403fff640 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166c13800 0x565166c15c20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbe57465fafa439 server_cookie=1fcdb85476cae833 in_seq=0 out_seq=0 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f83e37fe640 1 -- 172.21.15.49:0/2180210072 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565166afd960 con 0x565166c13800 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f83e37fe640 10 client.17247 handle_command_reply: tid=0 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 client.17247 shutdown 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 msgr2=0x565166b95080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005b60 tx=0x7f83ec06d610 comp rx=0 tx=0).stop 2024-03-16T08:24:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 shutdown_connections 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x565166c13800 0x565166c15c20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565166b9bda0 0x565166b955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565166b93710 0x565166b95080 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 --2- 172.21.15.49:0/2180210072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565166b92a30 0x565166ba41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 >> 172.21.15.49:0/2180210072 conn(0x565166b59c50 msgr2=0x565166b8c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f83e37fe640 0 client.17247 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 shutdown_connections 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 1 -- 172.21.15.49:0/2180210072 wait complete. 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.277+0000 7f840d1c4740 20 client.17247 trim_cache size 0 max 16384 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:07.284 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:24:07.285 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "74fdf62e-9711-408a-bbf8-cebfb3d924f6", 2024-03-16T08:24:07.285 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:24:07.285 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:07.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 msgr2=0x7f83f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 0x7f83f00a4e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00fc90 tx=0x7f83fc018000 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 msgr2=0x7f840419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e9f0 tx=0x7f83ec00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 shutdown_connections 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f83f00a2970 0x7f83f00a4e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f840410e180 0x7f840419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84040692e0 0x7f840419a7c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 --2- 172.21.15.49:0/2887655990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84040689c0 0x7f840419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 >> 172.21.15.49:0/2887655990 conn(0x7f84040777e0 msgr2=0x7f840410f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 shutdown_connections 2024-03-16T08:24:07.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.278+0000 7f840a5d4640 1 -- 172.21.15.49:0/2887655990 wait complete. 2024-03-16T08:24:07.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:24:07.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 -- 172.21.15.49:0/3368581103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 msgr2=0x7f477810b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 --2- 172.21.15.49:0/3368581103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f477810b820 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4768009900 tx=0x7f476802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 -- 172.21.15.49:0/3368581103 shutdown_connections 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 --2- 172.21.15.49:0/3368581103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f477810bdf0 0x7f477810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 --2- 172.21.15.49:0/3368581103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f477810b820 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 --2- 172.21.15.49:0/3368581103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4778102a20 0x7f4778102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 -- 172.21.15.49:0/3368581103 >> 172.21.15.49:0/3368581103 conn(0x7f4778076a60 msgr2=0x7f4778076e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 -- 172.21.15.49:0/3368581103 shutdown_connections 2024-03-16T08:24:07.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.399+0000 7f477e7b9640 1 -- 172.21.15.49:0/3368581103 wait complete. 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 Processor -- start 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 -- start start 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f477810bdf0 0x7f477819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111b80 con 0x7f4778103340 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111a00 con 0x7f4778102a20 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477e7b9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111d00 con 0x7f477810bdf0 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f47777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f4777fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f47777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49458/0 (socket says 172.21.15.49:49458) 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f4777fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:58436/0 (socket says 172.21.15.49:58436) 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f47777fe640 1 -- 172.21.15.49:0/774719482 learned_addr learned my addr 172.21.15.49:0/774719482 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.400+0000 7f477cd56640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f477810bdf0 0x7f477819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 -- 172.21.15.49:0/774719482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f477810bdf0 msgr2=0x7f477819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f477810bdf0 0x7f477819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 -- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 msgr2=0x7f47781985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 -- 172.21.15.49:0/774719482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477819d070 con 0x7f4778102a20 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f47777fe640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f4777fff640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f47757fa640 1 -- 172.21.15.49:0/774719482 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ccc0 con 0x7f4778102a20 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477819d360 con 0x7f4778102a20 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f47757fa640 1 -- 172.21.15.49:0/774719482 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476c0035f0 con 0x7f4778102a20 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f47757fa640 1 -- 172.21.15.49:0/774719482 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ce60 con 0x7f4778102a20 2024-03-16T08:24:07.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.401+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781005a0 con 0x7f4778102a20 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.403+0000 7f47757fa640 1 -- 172.21.15.49:0/774719482 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f476c0037a0 con 0x7f4778102a20 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.403+0000 7f47813a7740 1 Processor -- start 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 -- start start 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 0x55cc4dd8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dce7500 con 0x55cc4dd83a70 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dc8fbe0 con 0x55cc4dd8c650 2024-03-16T08:24:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47813a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dce35a0 con 0x55cc4dd83150 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 0x55cc4dd8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 0x55cc4dd8c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49466/0 (socket says 172.21.15.49:49466) 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f4777fff640 1 -- 172.21.15.49:0/1221521755 learned_addr learned my addr 172.21.15.49:0/1221521755 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:46126/0 (socket says 172.21.15.49:46126) 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47757fa640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 0x7f47600a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 -- 172.21.15.49:0/1221521755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd8c650 msgr2=0x55cc4dd8ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 -- 172.21.15.49:0/1221521755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 msgr2=0x55cc4dd8c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 0x55cc4dd8c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 -- 172.21.15.49:0/1221521755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4dca34b0 con 0x55cc4dd83150 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f47757fa640 1 -- 172.21.15.49:0/774719482 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f476c0cf0b0 con 0x7f4778102a20 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.404+0000 7f477cd56640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 0x7f47600a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f477cd56640 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4770005fd0 tx=0x7f4770010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2862c54437ad23d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f475e7fc640 1 -- 172.21.15.49:0/1221521755 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770020040 con 0x55cc4dd83150 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f475e7fc640 1 -- 172.21.15.49:0/1221521755 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f477000ba50 con 0x55cc4dd83150 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f477cd56640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 0x7f47600a4e30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4770026000 tx=0x7f4770027000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f475e7fc640 1 -- 172.21.15.49:0/1221521755 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770027950 con 0x55cc4dd83150 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 -- 172.21.15.49:0/1221521755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 msgr2=0x55cc4dd83530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4770005fd0 tx=0x7f4770010ab0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 -- 172.21.15.49:0/1221521755 shutdown_connections 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83a70 0x55cc4dd8c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 --2- 172.21.15.49:0/1221521755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83150 0x55cc4dd83530 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.405+0000 7f47813a7740 1 -- 172.21.15.49:0/1221521755 >> 172.21.15.49:0/1221521755 conn(0x55cc4dd7d880 msgr2=0x55cc4dd7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- 172.21.15.49:0/1221521755 shutdown_connections 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- 172.21.15.49:0/1221521755 wait complete. 2024-03-16T08:24:07.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- start start 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 Processor -- start 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd89c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dd8f440 con 0x55cc4dd83150 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dca3160 con 0x55cc4dd83a70 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cc4dd8f5c0 con 0x55cc4dd8c650 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47813a7740 10 client.0 _ll_register_callbacks cb 0x55cc4dc87150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f477cd56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f47777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd89c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49476/0 (socket says 172.21.15.49:49476) 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.406+0000 7f4777fff640 1 -- 172.21.15.49:0/3648497878 learned_addr learned my addr 172.21.15.49:0/3648497878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 -- 172.21.15.49:0/3648497878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 msgr2=0x55cc4dd8a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 -- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 msgr2=0x55cc4dd89c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd89c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 -- 172.21.15.49:0/3648497878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4dac57d0 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f4777fff640 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9c0 tx=0x7f476c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df396986e7b4a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47577fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c09b080 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f476c003b30 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f477000ba50 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f476c09be20 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47813a7740 10 client.17259 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cc4d9fb3f0 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.407+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f477000ba50 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f476c0d1110 con 0x55cc4dd83150 2024-03-16T08:24:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 10 client.17259 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 20 client.17259 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4dd9c720 0x55cc4dd9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 4 client.17259 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4dcf4a40 con 0x55cc4dd9c720 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47777fe640 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4dd9c720 0x55cc4dd9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47577fe640 10 client.17259 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.408+0000 7f47777fe640 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4dd9c720 0x55cc4dd9eb10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0b3daa65d0e1e3 server_cookie=2273026f4f7e27b8 in_seq=0 out_seq=0 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47577fe640 1 -- 172.21.15.49:0/3648497878 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cc4dcf4a40 con 0x55cc4dd9c720 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47577fe640 10 client.17259 handle_command_reply: tid=0 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 client.17259 shutdown 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 msgr2=0x55cc4dd89710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9c0 tx=0x7f476c009550 comp rx=0 tx=0).stop 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 shutdown_connections 2024-03-16T08:24:07.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4dd9c720 0x55cc4dd9eb10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd8a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd89c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47577fe640 0 client.17259 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 --2- 172.21.15.49:0/3648497878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd89710 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 >> 172.21.15.49:0/3648497878 conn(0x55cc4dd7d880 msgr2=0x55cc4dd7f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 shutdown_connections 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 1 -- 172.21.15.49:0/3648497878 wait complete. 2024-03-16T08:24:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.409+0000 7f47813a7740 20 client.17259 trim_cache size 0 max 16384 2024-03-16T08:24:07.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 Processor -- start 2024-03-16T08:24:07.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 -- start start 2024-03-16T08:24:07.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83150 0x55cc4dd7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83a70 0x55cc4dd981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47781ada90 con 0x55cc4dd8c650 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47781ab360 con 0x55cc4dd83150 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47813a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a91b0 con 0x55cc4dd83a70 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49492/0 (socket says 172.21.15.49:49492) 2024-03-16T08:24:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 -- 172.21.15.49:0/2936187964 learned_addr learned my addr 172.21.15.49:0/2936187964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f4777fff640 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83150 0x55cc4dd7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f47777fe640 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83a70 0x55cc4dd981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 -- 172.21.15.49:0/2936187964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83a70 msgr2=0x55cc4dd981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83a70 0x55cc4dd981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 -- 172.21.15.49:0/2936187964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83150 msgr2=0x55cc4dd7c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83150 0x55cc4dd7c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 -- 172.21.15.49:0/2936187964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4d9bb0f0 con 0x55cc4dd8c650 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f477cd56640 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f47700274c0 tx=0x7f4770027590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fde575e7d22beb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f475dffb640 1 -- 172.21.15.49:0/2936187964 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770020040 con 0x55cc4dd8c650 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f475dffb640 1 -- 172.21.15.49:0/2936187964 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4770027c10 con 0x55cc4dd8c650 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.427+0000 7f475dffb640 1 -- 172.21.15.49:0/2936187964 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477000bbd0 con 0x55cc4dd8c650 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- 172.21.15.49:0/2936187964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 msgr2=0x55cc4dda4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f47700274c0 tx=0x7f4770027590 comp rx=0 tx=0).stop 2024-03-16T08:24:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- 172.21.15.49:0/2936187964 shutdown_connections 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd8c650 0x55cc4dda4370 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd83a70 0x55cc4dd981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 --2- 172.21.15.49:0/2936187964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83150 0x55cc4dd7c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- 172.21.15.49:0/2936187964 >> 172.21.15.49:0/2936187964 conn(0x55cc4dd7d880 msgr2=0x55cc4dd896d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- 172.21.15.49:0/2936187964 shutdown_connections 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- 172.21.15.49:0/2936187964 wait complete. 2024-03-16T08:24:07.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.428+0000 7f47813a7740 1 -- start start 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 Processor -- start 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd86a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd86fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a9820 con 0x55cc4dd83150 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47781aac90 con 0x55cc4dd83a70 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a9ef0 con 0x55cc4dd8c650 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47813a7740 10 client.0 _ll_register_callbacks cb 0x7f476c003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49504/0 (socket says 172.21.15.49:49504) 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 -- 172.21.15.49:0/2211407128 learned_addr learned my addr 172.21.15.49:0/2211407128 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47777fe640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd86a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f477cd56640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd86fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 -- 172.21.15.49:0/2211407128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 msgr2=0x55cc4dd86fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd86fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 -- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 msgr2=0x55cc4dd86a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd86a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 -- 172.21.15.49:0/2211407128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4dcae9d0 con 0x55cc4dd83150 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f477cd56640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd86fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f4777fff640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f476c0a1d60 tx=0x7f476c0ccb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708a1c9e3492db5a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.429+0000 7f47577fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c09b080 con 0x55cc4dd83150 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8448+0+0 (secure 0 0 0) 0x7f476c09ad70 con 0x55cc4dd83150 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4770027c10 con 0x55cc4dd83150 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f476c09b4f0 con 0x55cc4dd83150 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 10 client.17265 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cc4de02d00 con 0x55cc4dd83150 2024-03-16T08:24:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4770027c10 con 0x55cc4dd83150 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f476c097470 con 0x55cc4dd83150 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 10 client.17265 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 20 client.17265 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4de03450 0x55cc4de05870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 4 client.17265 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc4dceed90 con 0x55cc4de03450 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.430+0000 7f47777fe640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4de03450 0x55cc4de05870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47577fe640 10 client.17265 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47777fe640 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4de03450 0x55cc4de05870 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365264c05d605c2a server_cookie=113801b62310f3ec in_seq=0 out_seq=0 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47577fe640 1 -- 172.21.15.49:0/2211407128 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cc4dceed90 con 0x55cc4de03450 2024-03-16T08:24:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47577fe640 10 client.17265 handle_command_reply: tid=0 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 client.17265 shutdown 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 msgr2=0x55cc4dd86530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f476c0a1d60 tx=0x7f476c0ccb00 comp rx=0 tx=0).stop 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 shutdown_connections 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55cc4de03450 0x55cc4de05870 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cc4dd8c650 0x55cc4dd86fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc4dd83a70 0x55cc4dd86a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47577fe640 0 client.17265 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 --2- 172.21.15.49:0/2211407128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc4dd83150 0x55cc4dd86530 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 >> 172.21.15.49:0/2211407128 conn(0x55cc4dd7d880 msgr2=0x55cc4dd7c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 shutdown_connections 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 1 -- 172.21.15.49:0/2211407128 wait complete. 2024-03-16T08:24:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.431+0000 7f47813a7740 20 client.17265 trim_cache size 0 max 16384 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "74fdf62e-9711-408a-bbf8-cebfb3d924f6": { 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "74fdf62e-9711-408a-bbf8-cebfb3d924f6", 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:24:07.445 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 msgr2=0x7f47600a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 0x7f47600a4e30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4770026000 tx=0x7f4770027000 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 msgr2=0x7f4778198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00edf0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 shutdown_connections 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f47600a2970 0x7f47600a4e30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f477810bdf0 0x7f477819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4778103340 0x7f47781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 --2- 172.21.15.49:0/774719482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4778102a20 0x7f4778198080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 >> 172.21.15.49:0/774719482 conn(0x7f4778076a60 msgr2=0x7f47780fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 shutdown_connections 2024-03-16T08:24:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:07.432+0000 7f477e7b9640 1 -- 172.21.15.49:0/774719482 wait complete. 2024-03-16T08:24:07.448 INFO:tasks.cephfs.filesystem:scrub status for tag:74fdf62e-9711-408a-bbf8-cebfb3d924f6 - {'path': '/', 'tag': '74fdf62e-9711-408a-bbf8-cebfb3d924f6', 'options': 'recursive,force'} 2024-03-16T08:24:08.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:07 smithi154 ceph-mon[36471]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:24:08.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:07 smithi049 ceph-mon[31942]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:24:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:07 smithi129 ceph-mon[36561]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:24:09.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:08 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:24:09.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:08 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:08 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:24:09.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:08 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:08 smithi154 ceph-mon[36471]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T08:24:09.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:08 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:24:09.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:08 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:08 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:24:09.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:08 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:08 smithi049 ceph-mon[31942]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T08:24:09.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:08 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:24:09.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:08 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:08 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:24:09.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:08 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:09.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:08 smithi129 ceph-mon[36561]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T08:24:10.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:10 smithi049 ceph-mon[31942]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-16T08:24:10.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:10 smithi049 ceph-mon[31942]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T08:24:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:10 smithi129 ceph-mon[36561]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-16T08:24:10.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:10 smithi129 ceph-mon[36561]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T08:24:10.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:10 smithi154 ceph-mon[36471]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-16T08:24:10.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:10 smithi154 ceph-mon[36471]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T08:24:12.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:12 smithi049 ceph-mon[31942]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-03-16T08:24:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:12 smithi129 ceph-mon[36561]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-03-16T08:24:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:12 smithi154 ceph-mon[36471]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-03-16T08:24:13.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:13 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:24:13.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:13 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:24:13.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:13 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:24:14.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:14 smithi049 ceph-mon[31942]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-03-16T08:24:14.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:14 smithi129 ceph-mon[36561]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-03-16T08:24:14.377 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:14 smithi154 ceph-mon[36471]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-03-16T08:24:16.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:16 smithi049 ceph-mon[31942]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:24:16.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:16 smithi129 ceph-mon[36561]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:24:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:16 smithi154 ceph-mon[36471]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:24:18.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:18 smithi049 ceph-mon[31942]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 230 B/s wr, 8 op/s 2024-03-16T08:24:18.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:18 smithi129 ceph-mon[36561]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 230 B/s wr, 8 op/s 2024-03-16T08:24:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:18 smithi154 ceph-mon[36471]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 230 B/s wr, 8 op/s 2024-03-16T08:24:20.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:20 smithi049 ceph-mon[31942]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-16T08:24:20.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:20 smithi129 ceph-mon[36561]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-16T08:24:20.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:20 smithi154 ceph-mon[36471]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-16T08:24:22.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:22 smithi049 ceph-mon[31942]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-16T08:24:22.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:22 smithi129 ceph-mon[36561]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-16T08:24:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:22 smithi154 ceph-mon[36471]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-16T08:24:24.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:24 smithi049 ceph-mon[31942]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:24:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:24 smithi129 ceph-mon[36561]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:24:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:24 smithi154 ceph-mon[36471]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-16T08:24:25.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:25 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:24:25.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:25 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:24:25.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:25 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:24:25.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:25 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:24:25.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:25 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:24:25.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:25 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:24:25.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:25 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:24:25.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:25 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:24:25.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:25 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:24:25.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:25 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:24:25.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:25 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:24:25.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:25 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:24:26.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:26 smithi049 ceph-mon[31942]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:24:26.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:26 smithi129 ceph-mon[36561]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:24:26.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:26 smithi154 ceph-mon[36471]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-16T08:24:28.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:28 smithi129 ceph-mon[36561]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:28.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:28 smithi154 ceph-mon[36471]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:28 smithi049 ceph-mon[31942]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:30.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:30 smithi129 ceph-mon[36561]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:30.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:30 smithi154 ceph-mon[36471]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:30.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:30 smithi049 ceph-mon[31942]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:32.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:32 smithi129 ceph-mon[36561]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:32.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:32 smithi154 ceph-mon[36471]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:32 smithi049 ceph-mon[31942]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:34.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:34 smithi129 ceph-mon[36561]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:34.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:34 smithi154 ceph-mon[36471]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:34.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:34 smithi049 ceph-mon[31942]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:36.340 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:36 smithi049 ceph-mon[31942]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:36.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:36 smithi129 ceph-mon[36561]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:36.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:36 smithi154 ceph-mon[36471]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:37.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:24:37.449 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:24:37.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/2635564562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8101010 msgr2=0x7f3db810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/2635564562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8101010 0x7f3db810d520 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009cd0 tx=0x7f3db002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/2635564562 shutdown_connections 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/2635564562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8101010 0x7f3db810d520 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/2635564562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8100650 0x7f3db8100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/2635564562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 0x7f3db8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/2635564562 >> 172.21.15.49:0/2635564562 conn(0x7f3db80777e0 msgr2=0x7f3db8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/2635564562 shutdown_connections 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.572+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/2635564562 wait complete. 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 Processor -- start 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 -- start start 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 0x7f3db81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 0x7f3db819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8111b70 con 0x7f3db8100650 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81119f0 con 0x7f3db8101010 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbfe3d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8111cf0 con 0x7f3db8106660 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59428/0 (socket says 172.21.15.49:59428) 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/1560458750 learned_addr learned my addr 172.21.15.49:0/1560458750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 0x7f3db81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 0x7f3db819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/1560458750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 msgr2=0x7f3db819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 0x7f3db819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/1560458750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 msgr2=0x7f3db81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 0x7f3db81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/1560458750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db819d030 con 0x7f3db8100650 2024-03-16T08:24:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.573+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 0x7f3db81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9f0 tx=0x7f3db400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3daeffd640 1 -- 172.21.15.49:0/1560458750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db400cd80 con 0x7f3db8100650 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db819d320 con 0x7f3db8100650 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3daeffd640 1 -- 172.21.15.49:0/1560458750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db4005500 con 0x7f3db8100650 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3daeffd640 1 -- 172.21.15.49:0/1560458750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4010690 con 0x7f3db8100650 2024-03-16T08:24:37.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.574+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81a4bb0 con 0x7f3db8100650 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.576+0000 7f3daeffd640 1 -- 172.21.15.49:0/1560458750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3db40046c0 con 0x7f3db8100650 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 Processor -- start 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 -- start start 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3daeffd640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 0x7f3da40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 0x55f01be82530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be8b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be8b650 0x55f01be8da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3daeffd640 1 -- 172.21.15.49:0/1560458750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f3db40cf1d0 con 0x7f3db8100650 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f01bde6500 con 0x55f01be82150 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f01bd8ebe0 con 0x55f01be8b650 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f01bde25a0 con 0x55f01be82a70 2024-03-16T08:24:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 0x7f3da40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 0x55f01be82530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dbd3d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be8b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 0x55f01be82530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59434/0 (socket says 172.21.15.49:59434) 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.577+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/4239308349 learned_addr learned my addr 172.21.15.49:0/4239308349 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be8b650 0x55f01be8da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/4239308349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be8b650 msgr2=0x55f01be8da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be8b650 0x55f01be8da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/4239308349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 msgr2=0x55f01be82530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 0x55f01be82530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/4239308349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01bda24b0 con 0x55f01be82a70 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 0x7f3da40a4e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005cc0 tx=0x7f3db002ce60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be8b110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009fa0 tx=0x7f3da80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270ee8c76b6e0b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3da3fff640 1 -- 172.21.15.49:0/4239308349 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8004020 con 0x55f01be82a70 2024-03-16T08:24:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3da3fff640 1 -- 172.21.15.49:0/4239308349 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da80041c0 con 0x55f01be82a70 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.578+0000 7f3da3fff640 1 -- 172.21.15.49:0/4239308349 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800f780 con 0x55f01be82a70 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- 172.21.15.49:0/4239308349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 msgr2=0x55f01be8b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be8b110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009fa0 tx=0x7f3da80118d0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- 172.21.15.49:0/4239308349 shutdown_connections 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be8b650 0x55f01be8da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be8b110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/4239308349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82150 0x55f01be82530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- 172.21.15.49:0/4239308349 >> 172.21.15.49:0/4239308349 conn(0x55f01be7c880 msgr2=0x55f01be7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- 172.21.15.49:0/4239308349 shutdown_connections 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- 172.21.15.49:0/4239308349 wait complete. 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.579+0000 7f3dc2a22740 1 -- start start 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 Processor -- start 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f01be8e440 con 0x55f01be8b650 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f01bda2160 con 0x55f01be82150 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f01be8e5c0 con 0x55f01be82a70 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dc2a22740 10 client.0 _ll_register_callbacks cb 0x55f01bd86150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbd3d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbe3da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:49030/0 (socket says 172.21.15.49:49030) 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbd3d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:55150/0 (socket says 172.21.15.49:55150) 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/3427726502 learned_addr learned my addr 172.21.15.49:0/3427726502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.580+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59436/0 (socket says 172.21.15.49:59436) 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/3427726502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 msgr2=0x55f01be88c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/3427726502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 msgr2=0x55f01be89170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 -- 172.21.15.49:0/3427726502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01bbc47d0 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3db40cc9f0 tx=0x7f3db409b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffbd99c7f2b6291 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40b3070 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f3db40a3b60 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da80041c0 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3db409b850 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dc2a22740 10 client.27158 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f01bafa3f0 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da80041c0 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.581+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3db409cb80 con 0x55f01be82150 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 10 client.27158 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:37.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 20 client.27158 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01be9b6a0 0x55f01be9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 4 client.27158 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f01bdf3a40 con 0x55f01be9b6a0 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01be9b6a0 0x55f01be9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01be9b6a0 0x55f01be9dad0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a120f40070653a77 server_cookie=1ec545355770c32e in_seq=0 out_seq=0 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.582+0000 7f3da17fa640 10 client.27158 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3da17fa640 1 -- 172.21.15.49:0/3427726502 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f01bdf3a40 con 0x55f01be9b6a0 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3da17fa640 10 client.27158 handle_command_reply: tid=0 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 client.27158 shutdown 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 msgr2=0x55f01be886f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3db40cc9f0 tx=0x7f3db409b750 comp rx=0 tx=0).stop 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 shutdown_connections 2024-03-16T08:24:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01be9b6a0 0x55f01be9dad0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be89170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82a70 0x55f01be88c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/3427726502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be886f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3da17fa640 0 client.27158 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 >> 172.21.15.49:0/3427726502 conn(0x55f01be7c880 msgr2=0x55f01be7e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 shutdown_connections 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 1 -- 172.21.15.49:0/3427726502 wait complete. 2024-03-16T08:24:37.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.583+0000 7f3dc2a22740 20 client.27158 trim_cache size 0 max 16384 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 Processor -- start 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 -- start start 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82150 0x55f01be8ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82a70 0x55f01be7cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81afce0 con 0x55f01be8b650 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ad5b0 con 0x55f01be82a70 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ab420 con 0x55f01be82150 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59442/0 (socket says 172.21.15.49:59442) 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 -- 172.21.15.49:0/1886381665 learned_addr learned my addr 172.21.15.49:0/1886381665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82150 0x55f01be8ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82a70 0x55f01be7cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 -- 172.21.15.49:0/1886381665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82150 msgr2=0x55f01be8ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82150 0x55f01be8ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 -- 172.21.15.49:0/1886381665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82a70 msgr2=0x55f01be7cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82a70 0x55f01be7cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.601+0000 7f3dbe3da640 1 -- 172.21.15.49:0/1886381665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01baba0f0 con 0x55f01be8b650 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3db002c700 tx=0x7f3db002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf17d21fed6bee74 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3da37fe640 1 -- 172.21.15.49:0/1886381665 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0039040 con 0x55f01be8b650 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3da37fe640 1 -- 172.21.15.49:0/1886381665 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db0052070 con 0x55f01be8b650 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3da37fe640 1 -- 172.21.15.49:0/1886381665 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db004d450 con 0x55f01be8b650 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 -- 172.21.15.49:0/1886381665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 msgr2=0x55f01be971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3db002c700 tx=0x7f3db002e050 comp rx=0 tx=0).stop 2024-03-16T08:24:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 -- 172.21.15.49:0/1886381665 shutdown_connections 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be8b650 0x55f01be971e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82a70 0x55f01be7cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/1886381665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be82150 0x55f01be8ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 -- 172.21.15.49:0/1886381665 >> 172.21.15.49:0/1886381665 conn(0x55f01be7c880 msgr2=0x55f01be7b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 -- 172.21.15.49:0/1886381665 shutdown_connections 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.602+0000 7f3dc2a22740 1 -- 172.21.15.49:0/1886381665 wait complete. 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 -- start start 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 Processor -- start 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be8b650 0x55f01be8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81aba70 con 0x55f01be82a70 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81acee0 con 0x55f01be82150 2024-03-16T08:24:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ac140 con 0x55f01be8b650 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dc2a22740 10 client.0 _ll_register_callbacks cb 0x7f3db400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dbd3d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dbdbd9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dbd3d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59458/0 (socket says 172.21.15.49:59458) 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/2596468819 learned_addr learned my addr 172.21.15.49:0/2596468819 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.603+0000 7f3dbe3da640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be8b650 0x55f01be8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/2596468819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be8b650 msgr2=0x55f01be8f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be8b650 0x55f01be8f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 msgr2=0x55f01be93840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be93840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 -- 172.21.15.49:0/2596468819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01bdad9d0 con 0x55f01be82a70 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dbd3d8640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3da8003f70 tx=0x7f3da8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740f05fbbc696e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3da17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da801d040 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f3da80035f0 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db0052070 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3da8035080 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dc2a22740 10 client.17283 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f01bf01620 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.604+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db0052070 con 0x55f01be82a70 2024-03-16T08:24:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3da802b250 con 0x55f01be82a70 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 10 client.17283 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 20 client.17283 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01bf01cb0 0x55f01bf040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 4 client.17283 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:24:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f01bdedd90 con 0x55f01bf01cb0 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01bf01cb0 0x55f01bf040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dbdbd9640 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01bf01cb0 0x55f01bf040a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e87aaa99f8d68a5 server_cookie=f0920e3d47855f2e in_seq=0 out_seq=0 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3da17fa640 10 client.17283 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3da17fa640 1 -- 172.21.15.49:0/2596468819 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f01bdedd90 con 0x55f01bf01cb0 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3da17fa640 10 client.17283 handle_command_reply: tid=0 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 1 client.17283 shutdown 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 msgr2=0x55f01be8ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.605+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3da8003f70 tx=0x7f3da8011e50 comp rx=0 tx=0).stop 2024-03-16T08:24:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 shutdown_connections 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f01bf01cb0 0x55f01bf040a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f01be8b650 0x55f01be8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01be82a70 0x55f01be8ebe0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 --2- 172.21.15.49:0/2596468819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f01be82150 0x55f01be93840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3da17fa640 0 client.17283 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 >> 172.21.15.49:0/2596468819 conn(0x55f01be7c880 msgr2=0x55f01be886b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 shutdown_connections 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 1 -- 172.21.15.49:0/2596468819 wait complete. 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dc2a22740 20 client.17283 trim_cache size 0 max 16384 2024-03-16T08:24:37.615 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:24:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:24:37.616 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 msgr2=0x7f3da40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 0x7f3da40a4e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005cc0 tx=0x7f3db002ce60 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 msgr2=0x7f3db8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9f0 tx=0x7f3db400edf0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 shutdown_connections 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3da40a2970 0x7f3da40a4e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db8106660 0x7f3db819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3db8101010 0x7f3db81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 --2- 172.21.15.49:0/1560458750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db8100650 0x7f3db8198090 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 >> 172.21.15.49:0/1560458750 conn(0x7f3db80777e0 msgr2=0x7f3db8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 shutdown_connections 2024-03-16T08:24:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.606+0000 7f3dbfe3d640 1 -- 172.21.15.49:0/1560458750 wait complete. 2024-03-16T08:24:37.619 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:24:37.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:24:37.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 -- 172.21.15.49:0/3007922892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f447806a830 msgr2=0x7f447810e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 --2- 172.21.15.49:0/3007922892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f447806a830 0x7f447810e630 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ca0 tx=0x7f447002c180 comp rx=0 tx=0).stop 2024-03-16T08:24:37.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 -- 172.21.15.49:0/3007922892 shutdown_connections 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 --2- 172.21.15.49:0/3007922892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f447806a830 0x7f447810e630 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 --2- 172.21.15.49:0/3007922892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4478069e90 0x7f447806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 --2- 172.21.15.49:0/3007922892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44781077e0 0x7f4478107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 -- 172.21.15.49:0/3007922892 >> 172.21.15.49:0/3007922892 conn(0x7f4478077780 msgr2=0x7f4478077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 -- 172.21.15.49:0/3007922892 shutdown_connections 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.726+0000 7f447ef1b640 1 -- 172.21.15.49:0/3007922892 wait complete. 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 Processor -- start 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 -- start start 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f447806a830 0x7f4478198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44781077e0 0x7f447819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111ba0 con 0x7f4478069e90 2024-03-16T08:24:37.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111a20 con 0x7f447806a830 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ef1b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4478111d20 con 0x7f44781077e0 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59482/0 (socket says 172.21.15.49:59482) 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 -- 172.21.15.49:0/4037725106 learned_addr learned my addr 172.21.15.49:0/4037725106 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f4477fff640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f447806a830 0x7f4478198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447d4b8640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44781077e0 0x7f447819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 -- 172.21.15.49:0/4037725106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44781077e0 msgr2=0x7f447819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44781077e0 0x7f447819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 -- 172.21.15.49:0/4037725106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f447806a830 msgr2=0x7f4478198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f447806a830 0x7f4478198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.727+0000 7f447ccb7640 1 -- 172.21.15.49:0/4037725106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447819cff0 con 0x7f4478069e90 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f447ccb7640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f446c00ea60 tx=0x7f446c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c00cde0 con 0x7f4478069e90 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446c0035f0 con 0x7f4478069e90 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447819d2e0 con 0x7f4478069e90 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c010670 con 0x7f4478069e90 2024-03-16T08:24:37.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.728+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44781a4b70 con 0x7f4478069e90 2024-03-16T08:24:37.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.729+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4478102f00 con 0x7f4478069e90 2024-03-16T08:24:37.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.730+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f446c0037a0 con 0x7f4478069e90 2024-03-16T08:24:37.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.731+0000 7f4475ffb640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 0x7f44680a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:37.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.731+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f446c0cf100 con 0x7f4478069e90 2024-03-16T08:24:37.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.731+0000 7f4477fff640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 0x7f44680a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:37.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.731+0000 7f4477fff640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 0x7f44680a4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44640095f0 tx=0x7f4464005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:37.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.734+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f446c09bf10 con 0x7f4478069e90 2024-03-16T08:24:37.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.958+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4478109ce0 con 0x7f4478069e90 2024-03-16T08:24:37.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.958+0000 7f4475ffb640 1 -- 172.21.15.49:0/4037725106 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f446c09b810 con 0x7f4478069e90 2024-03-16T08:24:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:24:37.960 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:24:37.960 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:24:37.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 msgr2=0x7f44680a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 0x7f44680a4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44640095f0 tx=0x7f4464005f90 comp rx=0 tx=0).stop 2024-03-16T08:24:37.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 msgr2=0x7f4478198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:37.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f446c00ea60 tx=0x7f446c00ef20 comp rx=0 tx=0).stop 2024-03-16T08:24:37.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 shutdown_connections 2024-03-16T08:24:37.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f44680a29f0 0x7f44680a4eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44781077e0 0x7f447819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f447806a830 0x7f4478198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 --2- 172.21.15.49:0/4037725106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4478069e90 0x7f4478198000 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.960+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 >> 172.21.15.49:0/4037725106 conn(0x7f4478077780 msgr2=0x7f44780ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.961+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 shutdown_connections 2024-03-16T08:24:37.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:37.961+0000 7f447ef1b640 1 -- 172.21.15.49:0/4037725106 wait complete. 2024-03-16T08:24:37.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- 172.21.15.49:0/136552108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c8107830 msgr2=0x7f25c8107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 --2- 172.21.15.49:0/136552108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c8107830 0x7f25c8107c10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009930 tx=0x7f25c402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- 172.21.15.49:0/136552108 shutdown_connections 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 --2- 172.21.15.49:0/136552108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c80693a0 0x7f25c810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 --2- 172.21.15.49:0/136552108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 0x7f25c8068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 --2- 172.21.15.49:0/136552108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c8107830 0x7f25c8107c10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- 172.21.15.49:0/136552108 >> 172.21.15.49:0/136552108 conn(0x7f25c80777e0 msgr2=0x7f25c8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- 172.21.15.49:0/136552108 shutdown_connections 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- 172.21.15.49:0/136552108 wait complete. 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 Processor -- start 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.084+0000 7f25cfb68640 1 -- start start 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 0x7f25c8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c8107830 0x7f25c819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111b80 con 0x7f25c80693a0 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111a00 con 0x7f25c8107830 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111d00 con 0x7f25c80689e0 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59504/0 (socket says 172.21.15.49:59504) 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 -- 172.21.15.49:0/3040824151 learned_addr learned my addr 172.21.15.49:0/3040824151 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25ce105640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c8107830 0x7f25c819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd904640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 0x7f25c8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 -- 172.21.15.49:0/3040824151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 msgr2=0x7f25c8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 0x7f25c8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 -- 172.21.15.49:0/3040824151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c8107830 msgr2=0x7f25c819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c8107830 0x7f25c819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 -- 172.21.15.49:0/3040824151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c819d000 con 0x7f25c80693a0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cd103640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25beffd640 1 -- 172.21.15.49:0/3040824151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800cd80 con 0x7f25c80693a0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25beffd640 1 -- 172.21.15.49:0/3040824151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b8005500 con 0x7f25c80693a0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25beffd640 1 -- 172.21.15.49:0/3040824151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8010690 con 0x7f25c80693a0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c819d2f0 con 0x7f25c80693a0 2024-03-16T08:24:38.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.085+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a4b80 con 0x7f25c80693a0 2024-03-16T08:24:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.087+0000 7f25beffd640 1 -- 172.21.15.49:0/3040824151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f25b80046c0 con 0x7f25c80693a0 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 Processor -- start 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 -- start start 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25beffd640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 0x7f25b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cecae77490 con 0x55cecaf13830 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cecae1fc90 con 0x55cecaf14150 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.088+0000 7f25d282a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cecae73530 con 0x55cecaf1cd30 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25beffd640 1 -- 172.21.15.49:0/3040824151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f25b80cf230 con 0x7f25c80693a0 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25ce105640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 0x7f25b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59508/0 (socket says 172.21.15.49:59508) 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd904640 1 -- 172.21.15.49:0/2794127664 learned_addr learned my addr 172.21.15.49:0/2794127664 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25ce105640 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 -- 172.21.15.49:0/2794127664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 msgr2=0x55cecaf1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf1f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 -- 172.21.15.49:0/2794127664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 msgr2=0x55cecaf13c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 -- 172.21.15.49:0/2794127664 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cecaeda500 con 0x55cecaf14150 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd904640 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25ce105640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 0x7f25b40a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005fd0 tx=0x7f25c0009e60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25cd103640 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf1c7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25b80cc9a0 tx=0x7f25b809b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ce1475300c187c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25b3fff640 1 -- 172.21.15.49:0/2794127664 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b809b8b0 con 0x55cecaf14150 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25b3fff640 1 -- 172.21.15.49:0/2794127664 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b809fa00 con 0x55cecaf14150 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.089+0000 7f25b3fff640 1 -- 172.21.15.49:0/2794127664 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8097f20 con 0x55cecaf14150 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- 172.21.15.49:0/2794127664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 msgr2=0x55cecaf1c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf1c7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25b80cc9a0 tx=0x7f25b809b3f0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- 172.21.15.49:0/2794127664 shutdown_connections 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf1c7f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 --2- 172.21.15.49:0/2794127664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- 172.21.15.49:0/2794127664 >> 172.21.15.49:0/2794127664 conn(0x55cecaf0df60 msgr2=0x55cecaf0e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- 172.21.15.49:0/2794127664 shutdown_connections 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- 172.21.15.49:0/2794127664 wait complete. 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.090+0000 7f25d282a740 1 -- start start 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 Processor -- start 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf14150 0x55cecaf1a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 0x55cecaf1a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cecaf1fb20 con 0x55cecaf13830 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cecae33230 con 0x55cecaf1cd30 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cecaf1fca0 con 0x55cecaf14150 2024-03-16T08:24:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25d282a740 10 client.0 _ll_register_callbacks cb 0x55cecae6f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59514/0 (socket says 172.21.15.49:59514) 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 -- 172.21.15.49:0/1071845596 learned_addr learned my addr 172.21.15.49:0/1071845596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25ce105640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 0x55cecaf1a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd103640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf14150 0x55cecaf1a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 -- 172.21.15.49:0/1071845596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf14150 msgr2=0x55cecaf1a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf14150 0x55cecaf1a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 -- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 msgr2=0x55cecaf1a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 0x55cecaf1a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25cd904640 1 -- 172.21.15.49:0/1071845596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cecae42100 con 0x55cecaf13830 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.091+0000 7f25ce105640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 0x55cecaf1a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25cd904640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f25c402c170 tx=0x7f25c402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f282853ee3ed8c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c402cab0 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f25c402cc50 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b809fa00 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25c40046c0 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 10 client.17301 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cecab8b3f0 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b809fa00 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25c404f070 con 0x55cecaf13830 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 10 client.17301 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 20 client.17301 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf2d240 0x55cecaf2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 4 client.17301 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.092+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cecae849d0 con 0x55cecaf2d240 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.093+0000 7f25cd103640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf2d240 0x55cecaf2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.093+0000 7f25cd103640 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf2d240 0x55cecaf2f630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3759568eb756528 server_cookie=a4e062ee7b7f5ac1 in_seq=0 out_seq=0 2024-03-16T08:24:38.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.093+0000 7f25b17fa640 10 client.17301 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25b17fa640 1 -- 172.21.15.49:0/1071845596 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cecae849d0 con 0x55cecaf2d240 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25b17fa640 10 client.17301 handle_command_reply: tid=0 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 client.17301 shutdown 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 msgr2=0x55cecaf19e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f25c402c170 tx=0x7f25c402c800 comp rx=0 tx=0).stop 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 shutdown_connections 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf2d240 0x55cecaf2f630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf1cd30 0x55cecaf1a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf14150 0x55cecaf1a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 --2- 172.21.15.49:0/1071845596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf19e50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25b17fa640 0 client.17301 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 >> 172.21.15.49:0/1071845596 conn(0x55cecaf0df60 msgr2=0x55cecaf0fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 shutdown_connections 2024-03-16T08:24:38.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 1 -- 172.21.15.49:0/1071845596 wait complete. 2024-03-16T08:24:38.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.094+0000 7f25d282a740 20 client.17301 trim_cache size 0 max 16384 2024-03-16T08:24:38.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.107+0000 7f25d282a740 1 Processor -- start 2024-03-16T08:24:38.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 -- start start 2024-03-16T08:24:38.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf34f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81afc80 con 0x55cecaf13830 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ad550 con 0x55cecaf14150 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25d282a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ab3c0 con 0x55cecaf1cd30 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59528/0 (socket says 172.21.15.49:59528) 2024-03-16T08:24:38.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 -- 172.21.15.49:0/3001530978 learned_addr learned my addr 172.21.15.49:0/3001530978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd103640 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25ce105640 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf34f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 -- 172.21.15.49:0/3001530978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 msgr2=0x55cecaf34f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf34f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 -- 172.21.15.49:0/3001530978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 msgr2=0x55cecaf0ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf0ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 -- 172.21.15.49:0/3001530978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cecaf13f80 con 0x55cecaf13830 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25cd904640 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f25c4004460 tx=0x7f25c402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee28fc4350f33c71 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.108+0000 7f25b37fe640 1 -- 172.21.15.49:0/3001530978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4055040 con 0x55cecaf13830 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25b37fe640 1 -- 172.21.15.49:0/3001530978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c402cab0 con 0x55cecaf13830 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25b37fe640 1 -- 172.21.15.49:0/3001530978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40053d0 con 0x55cecaf13830 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- 172.21.15.49:0/3001530978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 msgr2=0x55cecaf288c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f25c4004460 tx=0x7f25c402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- 172.21.15.49:0/3001530978 shutdown_connections 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf34f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 --2- 172.21.15.49:0/3001530978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf288c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- 172.21.15.49:0/3001530978 >> 172.21.15.49:0/3001530978 conn(0x55cecaf0df60 msgr2=0x55cecaf0c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- 172.21.15.49:0/3001530978 shutdown_connections 2024-03-16T08:24:38.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- 172.21.15.49:0/3001530978 wait complete. 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.109+0000 7f25d282a740 1 -- start start 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 Processor -- start 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf22400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf22940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81aba10 con 0x55cecaf13830 2024-03-16T08:24:38.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ace80 con 0x55cecaf14150 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ac0e0 con 0x55cecaf1cd30 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25d282a740 10 client.0 _ll_register_callbacks cb 0x7f25c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59532/0 (socket says 172.21.15.49:59532) 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 -- 172.21.15.49:0/282584541 learned_addr learned my addr 172.21.15.49:0/282584541 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25ce105640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf22940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd103640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf22400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 -- 172.21.15.49:0/282584541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 msgr2=0x55cecaf22940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf22940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 -- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 msgr2=0x55cecaf22400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf22400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 -- 172.21.15.49:0/282584541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cecac557d0 con 0x55cecaf13830 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25cd904640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f25c40057f0 tx=0x7f25c4005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b6fb2f777150cf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.110+0000 7f25b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4005050 con 0x55cecaf13830 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f25c40051f0 con 0x55cecaf13830 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c402cab0 con 0x55cecaf13830 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25c40053f0 con 0x55cecaf13830 2024-03-16T08:24:38.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 10 client.17307 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cecaf927a0 con 0x55cecaf13830 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c402cab0 con 0x55cecaf13830 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25c4069030 con 0x55cecaf13830 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 10 client.17307 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 10 client.17307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:24:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 20 client.17307 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf92e00 0x55cecaf95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 4 client.17307 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cecae7ed20 con 0x55cecaf92e00 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.111+0000 7f25cd103640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf92e00 0x55cecaf95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25cd103640 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf92e00 0x55cecaf95220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4227366d3031505 server_cookie=2c9d2f3654993a51 in_seq=0 out_seq=0 2024-03-16T08:24:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25b17fa640 10 client.17307 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25b17fa640 1 -- 172.21.15.49:0/282584541 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cecae7ed20 con 0x55cecaf92e00 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25b17fa640 10 client.17307 handle_command_reply: tid=0 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 client.17307 shutdown 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 msgr2=0x55cecaf21ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f25c40057f0 tx=0x7f25c4005950 comp rx=0 tx=0).stop 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 shutdown_connections 2024-03-16T08:24:38.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55cecaf92e00 0x55cecaf95220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cecaf1cd30 0x55cecaf22940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cecaf14150 0x55cecaf22400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 --2- 172.21.15.49:0/282584541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cecaf13830 0x55cecaf21ec0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25b17fa640 0 client.17307 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 >> 172.21.15.49:0/282584541 conn(0x55cecaf0df60 msgr2=0x55cecaf0cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 shutdown_connections 2024-03-16T08:24:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 1 -- 172.21.15.49:0/282584541 wait complete. 2024-03-16T08:24:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.112+0000 7f25d282a740 20 client.17307 trim_cache size 0 max 16384 2024-03-16T08:24:38.129 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:38.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 msgr2=0x7f25b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 0x7f25b40a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005fd0 tx=0x7f25c0009e60 comp rx=0 tx=0).stop 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 msgr2=0x7f25c81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800edf0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 shutdown_connections 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f25b40a2970 0x7f25b40a4e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25c8107830 0x7f25c819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25c80693a0 0x7f25c81985a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 --2- 172.21.15.49:0/3040824151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c80689e0 0x7f25c8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 >> 172.21.15.49:0/3040824151 conn(0x7f25c80777e0 msgr2=0x7f25c80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 shutdown_connections 2024-03-16T08:24:38.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.113+0000 7f25cfb68640 1 -- 172.21.15.49:0/3040824151 wait complete. 2024-03-16T08:24:38.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:24:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 -- 172.21.15.49:0/3173817916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 msgr2=0x7f4654100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 --2- 172.21.15.49:0/3173817916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654100ad0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4650009930 tx=0x7f465002bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 -- 172.21.15.49:0/3173817916 shutdown_connections 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 --2- 172.21.15.49:0/3173817916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654101010 0x7f465410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 --2- 172.21.15.49:0/3173817916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654100ad0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 --2- 172.21.15.49:0/3173817916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654106660 0x7f4654106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 -- 172.21.15.49:0/3173817916 >> 172.21.15.49:0/3173817916 conn(0x7f46540777e0 msgr2=0x7f4654077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 -- 172.21.15.49:0/3173817916 shutdown_connections 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.245+0000 7f465bebf640 1 -- 172.21.15.49:0/3173817916 wait complete. 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 Processor -- start 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 -- start start 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654101010 0x7f46541985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654106660 0x7f465419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4654111bc0 con 0x7f4654100650 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4654111a40 con 0x7f4654101010 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465bebf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4654111d40 con 0x7f4654106660 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59548/0 (socket says 172.21.15.49:59548) 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f4659c5b640 1 -- 172.21.15.49:0/3690046170 learned_addr learned my addr 172.21.15.49:0/3690046170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465945a640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654101010 0x7f46541985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.246+0000 7f465a45c640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654106660 0x7f465419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 -- 172.21.15.49:0/3690046170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654106660 msgr2=0x7f465419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654106660 0x7f465419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 -- 172.21.15.49:0/3690046170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654101010 msgr2=0x7f46541985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654101010 0x7f46541985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 -- 172.21.15.49:0/3690046170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465419d050 con 0x7f4654100650 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f4659c5b640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f464400e9f0 tx=0x7f464400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f464affd640 1 -- 172.21.15.49:0/3690046170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464400cd80 con 0x7f4654100650 2024-03-16T08:24:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465419d340 con 0x7f4654100650 2024-03-16T08:24:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f464affd640 1 -- 172.21.15.49:0/3690046170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4644010910 con 0x7f4654100650 2024-03-16T08:24:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f464affd640 1 -- 172.21.15.49:0/3690046170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440188f0 con 0x7f4654100650 2024-03-16T08:24:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.247+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46541a4bd0 con 0x7f4654100650 2024-03-16T08:24:38.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.249+0000 7f464affd640 1 -- 172.21.15.49:0/3690046170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4644010ac0 con 0x7f4654100650 2024-03-16T08:24:38.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.249+0000 7f465eb81740 1 Processor -- start 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 -- start start 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 0x55b2b69f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b69fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b6955490 con 0x55b2b69fad30 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2b68fdc90 con 0x55b2b69f2150 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465eb81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b2b6951530 con 0x55b2b69f1830 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f464affd640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 0x7f46400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465945a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465945a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59552/0 (socket says 172.21.15.49:59552) 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465945a640 1 -- 172.21.15.49:0/3522875913 learned_addr learned my addr 172.21.15.49:0/3522875913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f4659c5b640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b69fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465a45c640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 0x55b2b69f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f465a45c640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 0x7f46400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.250+0000 7f464affd640 1 -- 172.21.15.49:0/3690046170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f46440cf150 con 0x7f4654100650 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 -- 172.21.15.49:0/3522875913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 msgr2=0x55b2b69f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 0x55b2b69f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 -- 172.21.15.49:0/3522875913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 msgr2=0x55b2b69fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b69fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 -- 172.21.15.49:0/3522875913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b69b8500 con 0x55b2b69fad30 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f4659c5b640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b69fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465a45c640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 0x55b2b69f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465a45c640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 0x7f46400a4e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464c005c40 tx=0x7f464c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f465945a640 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f4650009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ee86d943f5a931 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f463ffff640 1 -- 172.21.15.49:0/3522875913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650045070 con 0x55b2b69fad30 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f463ffff640 1 -- 172.21.15.49:0/3522875913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465002c650 con 0x55b2b69fad30 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.251+0000 7f463ffff640 1 -- 172.21.15.49:0/3522875913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650004a20 con 0x55b2b69fad30 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- 172.21.15.49:0/3522875913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 msgr2=0x55b2b69fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f4650009560 comp rx=0 tx=0).stop 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- 172.21.15.49:0/3522875913 shutdown_connections 2024-03-16T08:24:38.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69fd120 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b69fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 --2- 172.21.15.49:0/3522875913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f1830 0x55b2b69f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- 172.21.15.49:0/3522875913 >> 172.21.15.49:0/3522875913 conn(0x55b2b69ebf60 msgr2=0x55b2b69ec340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- 172.21.15.49:0/3522875913 shutdown_connections 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- 172.21.15.49:0/3522875913 wait complete. 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 -- start start 2024-03-16T08:24:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.252+0000 7f465eb81740 1 Processor -- start 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f1830 0x55b2b69f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b69fdac0 con 0x55b2b69fad30 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2b6911230 con 0x55b2b69f1830 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b2b69fdc40 con 0x55b2b69f2150 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465eb81740 10 client.0 _ll_register_callbacks cb 0x55b2b694d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59558/0 (socket says 172.21.15.49:59558) 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 -- 172.21.15.49:0/2112755201 learned_addr learned my addr 172.21.15.49:0/2112755201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465945a640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f1830 0x55b2b69f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 -- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 msgr2=0x55b2b69f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 -- 172.21.15.49:0/2112755201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f1830 msgr2=0x55b2b69f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f1830 0x55b2b69f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 -- 172.21.15.49:0/2112755201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b6920100 con 0x55b2b69fad30 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f465a45c640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f464c019180 tx=0x7f464c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe5399644bb6dd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f463d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.253+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c02d060 con 0x55b2b69fad30 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f464c0047b0 con 0x55b2b69fad30 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465002c650 con 0x55b2b69fad30 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f464c020460 con 0x55b2b69fad30 2024-03-16T08:24:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 10 client.17319 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b2b66693f0 con 0x55b2b69fad30 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465002c650 con 0x55b2b69fad30 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f464c027030 con 0x55b2b69fad30 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 10 client.17319 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 20 client.17319 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a0b110 0x55b2b6a0d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 4 client.17319 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b69629d0 con 0x55b2b6a0b110 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.254+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a0b110 0x55b2b6a0d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f463d7fa640 10 client.17319 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a0b110 0x55b2b6a0d560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32db9a4d1370b8b9 server_cookie=d1ab2ed9b4783eed in_seq=0 out_seq=0 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f463d7fa640 1 -- 172.21.15.49:0/2112755201 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b2b69629d0 con 0x55b2b6a0b110 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f463d7fa640 10 client.17319 handle_command_reply: tid=0 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f465eb81740 1 client.17319 shutdown 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 msgr2=0x55b2b69f87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.255+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f464c019180 tx=0x7f464c004a40 comp rx=0 tx=0).stop 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 shutdown_connections 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a0b110 0x55b2b6a0d560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69fad30 0x55b2b69f87d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f463d7fa640 0 client.17319 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 --2- 172.21.15.49:0/2112755201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f1830 0x55b2b69f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 >> 172.21.15.49:0/2112755201 conn(0x55b2b69ebf60 msgr2=0x55b2b69edcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 shutdown_connections 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 1 -- 172.21.15.49:0/2112755201 wait complete. 2024-03-16T08:24:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.256+0000 7f465eb81740 20 client.17319 trim_cache size 0 max 16384 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 Processor -- start 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 -- start start 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b6a06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69fad30 0x55b2b6a12e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46541afc60 con 0x55b2b69f1830 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46541ad530 con 0x55b2b69f2150 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465eb81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f46541ab3a0 con 0x55b2b69fad30 2024-03-16T08:24:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59574/0 (socket says 172.21.15.49:59574) 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 -- 172.21.15.49:0/2226419479 learned_addr learned my addr 172.21.15.49:0/2226419479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465a45c640 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69fad30 0x55b2b6a12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f465945a640 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b6a06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 -- 172.21.15.49:0/2226419479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69fad30 msgr2=0x55b2b6a12e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69fad30 0x55b2b6a12e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 -- 172.21.15.49:0/2226419479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 msgr2=0x55b2b6a06890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b6a06890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.269+0000 7f4659c5b640 1 -- 172.21.15.49:0/2226419479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b67337d0 con 0x55b2b69f1830 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f4659c5b640 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f46440cc970 tx=0x7f4644096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ededbbe26de0fd2a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f463f7fe640 1 -- 172.21.15.49:0/2226419479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644099c40 con 0x55b2b69f1830 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f463f7fe640 1 -- 172.21.15.49:0/2226419479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46440ccb40 con 0x55b2b69f1830 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f463f7fe640 1 -- 172.21.15.49:0/2226419479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464406d370 con 0x55b2b69f1830 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 -- 172.21.15.49:0/2226419479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 msgr2=0x55b2b69fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f46440cc970 tx=0x7f4644096040 comp rx=0 tx=0).stop 2024-03-16T08:24:38.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 -- 172.21.15.49:0/2226419479 shutdown_connections 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69fad30 0x55b2b6a12e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69f2150 0x55b2b6a06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 --2- 172.21.15.49:0/2226419479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b69fa480 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 -- 172.21.15.49:0/2226419479 >> 172.21.15.49:0/2226419479 conn(0x55b2b69ebf60 msgr2=0x55b2b69edbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 -- 172.21.15.49:0/2226419479 shutdown_connections 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.270+0000 7f465eb81740 1 -- 172.21.15.49:0/2226419479 wait complete. 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 -- start start 2024-03-16T08:24:38.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 Processor -- start 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69fad30 0x55b2b69f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46541ab9f0 con 0x55b2b69f1830 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46541ace60 con 0x55b2b69fad30 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f46541ac0c0 con 0x55b2b69f2150 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465eb81740 10 client.0 _ll_register_callbacks cb 0x7f464c019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f4659c5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59586/0 (socket says 172.21.15.49:59586) 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f4659c5b640 1 -- 172.21.15.49:0/938478663 learned_addr learned my addr 172.21.15.49:0/938478663 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465945a640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.271+0000 7f465a45c640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69fad30 0x55b2b69f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 -- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 msgr2=0x55b2b69f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 -- 172.21.15.49:0/938478663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69fad30 msgr2=0x55b2b69f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69fad30 0x55b2b69f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 -- 172.21.15.49:0/938478663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b69f5510 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f4659c5b640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4644097ce0 tx=0x7f4644097c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f471c27cdf8fa13 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644099c40 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f46440afbb0 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46440ccb40 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f46440a3f70 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 10 client.17325 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b2b6a71560 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46440ccb40 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4644097660 con 0x55b2b69f1830 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 10 client.17325 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 10 client.17325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:24:38.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 20 client.17325 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a71ce0 0x55b2b6a74100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.272+0000 7f465eb81740 4 client.17325 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2b695cd20 con 0x55b2b6a71ce0 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465945a640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a71ce0 0x55b2b6a74100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f463d7fa640 10 client.17325 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465945a640 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a71ce0 0x55b2b6a74100 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88817e5f391b7c06 server_cookie=8d12f99c7d3998fd in_seq=0 out_seq=0 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f463d7fa640 1 -- 172.21.15.49:0/938478663 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2b695cd20 con 0x55b2b6a71ce0 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f463d7fa640 10 client.17325 handle_command_reply: tid=0 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 client.17325 shutdown 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 msgr2=0x55b2b6a03270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4644097ce0 tx=0x7f4644097c50 comp rx=0 tx=0).stop 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 shutdown_connections 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b2b6a71ce0 0x55b2b6a74100 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b69fad30 0x55b2b69f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2b69f2150 0x55b2b69f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f463d7fa640 0 client.17325 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:24:38.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 --2- 172.21.15.49:0/938478663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b69f1830 0x55b2b6a03270 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.273+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 >> 172.21.15.49:0/938478663 conn(0x55b2b69ebf60 msgr2=0x55b2b69edbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 shutdown_connections 2024-03-16T08:24:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465eb81740 1 -- 172.21.15.49:0/938478663 wait complete. 2024-03-16T08:24:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465eb81740 20 client.17325 trim_cache size 0 max 16384 2024-03-16T08:24:38.281 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 msgr2=0x7f46400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 0x7f46400a4e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464c005c40 tx=0x7f464c009500 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 msgr2=0x7f4654198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f464400e9f0 tx=0x7f464400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 shutdown_connections 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f46400a2970 0x7f46400a4e30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4654106660 0x7f465419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4654101010 0x7f46541985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 --2- 172.21.15.49:0/3690046170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4654100650 0x7f4654198060 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 >> 172.21.15.49:0/3690046170 conn(0x7f46540777e0 msgr2=0x7f4654102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 shutdown_connections 2024-03-16T08:24:38.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.274+0000 7f465bebf640 1 -- 172.21.15.49:0/3690046170 wait complete. 2024-03-16T08:24:38.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:38 smithi049 ceph-mon[31942]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:38.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:38 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4037725106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:38.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:24:38.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:38 smithi129 ceph-mon[36561]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:38.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:38 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4037725106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:38.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 -- 172.21.15.49:0/1177323189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 msgr2=0x7f40b4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 --2- 172.21.15.49:0/1177323189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b4101c00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009930 tx=0x7f40a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 -- 172.21.15.49:0/1177323189 shutdown_connections 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 --2- 172.21.15.49:0/1177323189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 0x7f40b410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 --2- 172.21.15.49:0/1177323189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b4101c00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 --2- 172.21.15.49:0/1177323189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b4107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 -- 172.21.15.49:0/1177323189 >> 172.21.15.49:0/1177323189 conn(0x7f40b40fd210 msgr2=0x7f40b40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 -- 172.21.15.49:0/1177323189 shutdown_connections 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.394+0000 7f40bc509640 1 -- 172.21.15.49:0/1177323189 wait complete. 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 Processor -- start 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 -- start start 2024-03-16T08:24:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 0x7f40b4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4111b60 con 0x7f40b4101780 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41119e0 con 0x7f40b4107790 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40bc509640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4111ce0 con 0x7f40b4102140 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59598/0 (socket says 172.21.15.49:59598) 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40baaa6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3809641121 learned_addr learned my addr 172.21.15.49:0/3809641121 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.395+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 0x7f40b4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 -- 172.21.15.49:0/3809641121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 msgr2=0x7f40b4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 0x7f40b4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 -- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 msgr2=0x7f40b41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 -- 172.21.15.49:0/3809641121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b419d080 con 0x7f40b4107790 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40baaa6640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b419c950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009fd0 tx=0x7f40ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40ab7fe640 1 -- 172.21.15.49:0/3809641121 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac00ce20 con 0x7f40b4107790 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b419d370 con 0x7f40b4107790 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40ab7fe640 1 -- 172.21.15.49:0/3809641121 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac005020 con 0x7f40b4107790 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40ab7fe640 1 -- 172.21.15.49:0/3809641121 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac010720 con 0x7f40b4107790 2024-03-16T08:24:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.396+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a4c00 con 0x7f40b4107790 2024-03-16T08:24:38.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40ab7fe640 1 -- 172.21.15.49:0/3809641121 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f40ac005540 con 0x7f40b4107790 2024-03-16T08:24:38.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 Processor -- start 2024-03-16T08:24:38.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 -- start start 2024-03-16T08:24:38.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04391c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc0439a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc0439ad30 0x55fc0439d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc042f5490 con 0x55fc04391830 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fc0429dc90 con 0x55fc0439ad30 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc042f1530 con 0x55fc04392150 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04391c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04391c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59604/0 (socket says 172.21.15.49:59604) 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40b9aa4640 1 -- 172.21.15.49:0/2599510516 learned_addr learned my addr 172.21.15.49:0/2599510516 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc0439a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.399+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc0439ad30 0x55fc0439d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 -- 172.21.15.49:0/2599510516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc0439ad30 msgr2=0x55fc0439d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc0439ad30 0x55fc0439d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ab7fe640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 0x7f40a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 -- 172.21.15.49:0/2599510516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 msgr2=0x55fc04391c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04391c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 -- 172.21.15.49:0/2599510516 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc04358500 con 0x55fc04392150 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40baaa6640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 0x7f40a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc0439a7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005d00 tx=0x7f40b0011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8846c159999092c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40ab7fe640 1 -- 172.21.15.49:0/3809641121 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f40ac0cf530 con 0x7f40b4107790 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40a8bfe640 1 -- 172.21.15.49:0/2599510516 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0005020 con 0x55fc04392150 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40a8bfe640 1 -- 172.21.15.49:0/2599510516 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b00047a0 con 0x55fc04392150 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.400+0000 7f40a8bfe640 1 -- 172.21.15.49:0/2599510516 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000f8b0 con 0x55fc04392150 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- 172.21.15.49:0/2599510516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 msgr2=0x55fc0439a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40baaa6640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 0x7f40a00a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cd2a0 tx=0x7f40ac013700 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc0439a7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005d00 tx=0x7f40b0011950 comp rx=0 tx=0).stop 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- 172.21.15.49:0/2599510516 shutdown_connections 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc0439ad30 0x55fc0439d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc0439a7f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/2599510516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04391c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- 172.21.15.49:0/2599510516 >> 172.21.15.49:0/2599510516 conn(0x55fc0438bf60 msgr2=0x55fc0438c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- 172.21.15.49:0/2599510516 shutdown_connections 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- 172.21.15.49:0/2599510516 wait complete. 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 -- start start 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 Processor -- start 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.401+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc043987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc0439db20 con 0x55fc04391830 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fc042b1230 con 0x55fc04392150 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc0439dca0 con 0x55fc0439ad30 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40bf1d5740 10 client.0 _ll_register_callbacks cb 0x55fc042ed010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59618/0 (socket says 172.21.15.49:59618) 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40b9aa4640 1 -- 172.21.15.49:0/3082721069 learned_addr learned my addr 172.21.15.49:0/3082721069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:49208/0 (socket says 172.21.15.49:49208) 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc043987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3082721069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 msgr2=0x55fc043987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc043987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3082721069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 msgr2=0x55fc04397d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3082721069 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc042c0100 con 0x55fc04392150 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f40b001b040 tx=0x7f40b0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc42caba190ed7b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f409dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0028020 con 0x55fc04392150 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f40b0016040 con 0x55fc04392150 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.402+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b00047a0 con 0x55fc04392150 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f40b0033050 con 0x55fc04392150 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 10 client.27218 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc040093f0 con 0x55fc04392150 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b00047a0 con 0x55fc04392150 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f40b0003890 con 0x55fc04392150 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 10 client.27218 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 10 client.27218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 20 client.27218 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc043ab100 0x55fc043ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 4 client.27218 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc043029d0 con 0x55fc043ab100 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.403+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc043ab100 0x55fc043ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.404+0000 7f409dffb640 10 client.27218 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.404+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc043ab100 0x55fc043ad4f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f30f36490b47270 server_cookie=150edb2a3072710a in_seq=0 out_seq=0 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.404+0000 7f409dffb640 1 -- 172.21.15.49:0/3082721069 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fc043029d0 con 0x55fc043ab100 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.404+0000 7f409dffb640 10 client.27218 handle_command_reply: tid=0 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.404+0000 7f40bf1d5740 1 client.27218 shutdown 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 msgr2=0x55fc043982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f40b001b040 tx=0x7f40b0004270 comp rx=0 tx=0).stop 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 shutdown_connections 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc043ab100 0x55fc043ad4f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc043987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f409dffb640 0 client.27218 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04392150 0x55fc043982b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3082721069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04391830 0x55fc04397d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 >> 172.21.15.49:0/3082721069 conn(0x55fc0438bf60 msgr2=0x55fc0438dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 shutdown_connections 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3082721069 wait complete. 2024-03-16T08:24:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.405+0000 7f40bf1d5740 20 client.27218 trim_cache size 0 max 16384 2024-03-16T08:24:38.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 Processor -- start 2024-03-16T08:24:38.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 -- start start 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc0439a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc043a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41afd20 con 0x55fc0439ad30 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ad5f0 con 0x55fc04391830 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.418+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ab460 con 0x55fc04392150 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59624/0 (socket says 172.21.15.49:59624) 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 -- 172.21.15.49:0/526310498 learned_addr learned my addr 172.21.15.49:0/526310498 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc043a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc0439a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 -- 172.21.15.49:0/526310498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 msgr2=0x55fc043a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc043a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 -- 172.21.15.49:0/526310498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 msgr2=0x55fc0439a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc0439a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 -- 172.21.15.49:0/526310498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc043ad860 con 0x55fc0439ad30 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f40a4006010 tx=0x7f40a402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf1383803804d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f409ffff640 1 -- 172.21.15.49:0/526310498 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40503d0 con 0x55fc0439ad30 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f409ffff640 1 -- 172.21.15.49:0/526310498 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a4004ba0 con 0x55fc0439ad30 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.419+0000 7f409ffff640 1 -- 172.21.15.49:0/526310498 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40052d0 con 0x55fc0439ad30 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- 172.21.15.49:0/526310498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 msgr2=0x55fc043b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f40a4006010 tx=0x7f40a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- 172.21.15.49:0/526310498 shutdown_connections 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc0439ad30 0x55fc043b2cd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc04392150 0x55fc043a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/526310498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc0439a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- 172.21.15.49:0/526310498 >> 172.21.15.49:0/526310498 conn(0x55fc0438bf60 msgr2=0x55fc043930b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- 172.21.15.49:0/526310498 shutdown_connections 2024-03-16T08:24:38.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- 172.21.15.49:0/526310498 wait complete. 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- start start 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 Processor -- start 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc043a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc0439e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41abab0 con 0x55fc04392150 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41acf20 con 0x55fc04391830 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ac180 con 0x55fc0439ad30 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40bf1d5740 10 client.0 _ll_register_callbacks cb 0x7f40ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40ba2a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59636/0 (socket says 172.21.15.49:59636) 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.420+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3565495180 learned_addr learned my addr 172.21.15.49:0/3565495180 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc043a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc0439e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3565495180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 msgr2=0x55fc0439e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc0439e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 msgr2=0x55fc043a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc043a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 -- 172.21.15.49:0/3565495180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc040d37d0 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc043a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40ba2a5640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f40b00098e0 tx=0x7f40b0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1b4aa24d1517ee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f409dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b003c060 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f40b00035f0 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a4004ba0 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f40b0028040 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40bf1d5740 10 client.17343 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc04410d30 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.421+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a4004ba0 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f40b000fcc0 con 0x55fc04392150 2024-03-16T08:24:38.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 10 client.17343 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 20 client.17343 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc044114e0 0x55fc04413900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 4 client.17343 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc042fcd20 con 0x55fc044114e0 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc044114e0 0x55fc04413900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f409dffb640 10 client.17343 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40b9aa4640 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc044114e0 0x55fc04413900 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82e7a57060cac43 server_cookie=707f077d058edcc5 in_seq=0 out_seq=0 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f409dffb640 1 -- 172.21.15.49:0/3565495180 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc042fcd20 con 0x55fc044114e0 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f409dffb640 10 client.17343 handle_command_reply: tid=0 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 1 client.17343 shutdown 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 msgr2=0x55fc043a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.422+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f40b00098e0 tx=0x7f40b0005f10 comp rx=0 tx=0).stop 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 shutdown_connections 2024-03-16T08:24:38.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fc044114e0 0x55fc04413900 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc0439ad30 0x55fc0439e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc04392150 0x55fc043a3320 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f409dffb640 0 client.17343 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 --2- 172.21.15.49:0/3565495180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fc04391830 0x55fc043a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 >> 172.21.15.49:0/3565495180 conn(0x55fc0438bf60 msgr2=0x55fc04397d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 shutdown_connections 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 1 -- 172.21.15.49:0/3565495180 wait complete. 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bf1d5740 20 client.17343 trim_cache size 0 max 16384 2024-03-16T08:24:38.430 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 msgr2=0x7f40a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 0x7f40a00a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cd2a0 tx=0x7f40ac013700 comp rx=0 tx=0).stop 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 msgr2=0x7f40b419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b419c950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009fd0 tx=0x7f40ac00ef10 comp rx=0 tx=0).stop 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 shutdown_connections 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f40a00a2970 0x7f40a00a4e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b4107790 0x7f40b419c950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40b4102140 0x7f40b4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 --2- 172.21.15.49:0/3809641121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40b4101780 0x7f40b41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:38.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 >> 172.21.15.49:0/3809641121 conn(0x7f40b40fd210 msgr2=0x7f40b40fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:38.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 shutdown_connections 2024-03-16T08:24:38.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:38.423+0000 7f40bc509640 1 -- 172.21.15.49:0/3809641121 wait complete. 2024-03-16T08:24:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:38 smithi154 ceph-mon[36471]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:38 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4037725106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:24:39.436 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:24:39.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:24:39.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:24:39.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 -- 172.21.15.49:0/1154180767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 msgr2=0x7f00e8101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 --2- 172.21.15.49:0/1154180767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8101c00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009930 tx=0x7f00dc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 -- 172.21.15.49:0/1154180767 shutdown_connections 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 --2- 172.21.15.49:0/1154180767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8102140 0x7f00e810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 --2- 172.21.15.49:0/1154180767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8101c00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 --2- 172.21.15.49:0/1154180767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8107790 0x7f00e8107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 -- 172.21.15.49:0/1154180767 >> 172.21.15.49:0/1154180767 conn(0x7f00e80fd210 msgr2=0x7f00e80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 -- 172.21.15.49:0/1154180767 shutdown_connections 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.560+0000 7f00eefcd640 1 -- 172.21.15.49:0/1154180767 wait complete. 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 Processor -- start 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 -- start start 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8102140 0x7f00e8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8107790 0x7f00e819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111ba0 con 0x7f00e8101780 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111a20 con 0x7f00e8107790 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00eefcd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111d20 con 0x7f00e8102140 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59656/0 (socket says 172.21.15.49:59656) 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 -- 172.21.15.49:0/4265473200 learned_addr learned my addr 172.21.15.49:0/4265473200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ed56a640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8107790 0x7f00e819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00e7fff640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8102140 0x7f00e8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 -- 172.21.15.49:0/4265473200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8102140 msgr2=0x7f00e8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8102140 0x7f00e8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 -- 172.21.15.49:0/4265473200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8107790 msgr2=0x7f00e819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8107790 0x7f00e819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 -- 172.21.15.49:0/4265473200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819d0f0 con 0x7f00e8101780 2024-03-16T08:24:39.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.561+0000 7f00ecd69640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eb90 tx=0x7f00e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.562+0000 7f00e5ffb640 1 -- 172.21.15.49:0/4265473200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0004d20 con 0x7f00e8101780 2024-03-16T08:24:39.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.562+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e819d3e0 con 0x7f00e8101780 2024-03-16T08:24:39.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.562+0000 7f00e5ffb640 1 -- 172.21.15.49:0/4265473200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e000ee90 con 0x7f00e8101780 2024-03-16T08:24:39.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.562+0000 7f00e5ffb640 1 -- 172.21.15.49:0/4265473200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0010830 con 0x7f00e8101780 2024-03-16T08:24:39.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.562+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a4c70 con 0x7f00e8101780 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.564+0000 7f00e5ffb640 1 -- 172.21.15.49:0/4265473200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f00e0005020 con 0x7f00e8101780 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.564+0000 7f00f1bb9740 1 Processor -- start 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 -- start start 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 0x55dc0bac8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bad1da0 0x55dc0bad4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc0ba2c0d0 con 0x55dc0bac9710 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc0b9d4c70 con 0x55dc0bac8a30 2024-03-16T08:24:39.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc0ba28170 con 0x55dc0bad1da0 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00e5ffb640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 0x7f00d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ecd69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 0x55dc0bac8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59658/0 (socket says 172.21.15.49:59658) 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 -- 172.21.15.49:0/1729685014 learned_addr learned my addr 172.21.15.49:0/1729685014 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00e7fff640 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bad1da0 0x55dc0bad4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 0x7f00d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00e5ffb640 1 -- 172.21.15.49:0/4265473200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f00e00cf400 con 0x7f00e8101780 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 -- 172.21.15.49:0/1729685014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bad1da0 msgr2=0x55dc0bad4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bad1da0 0x55dc0bad4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 -- 172.21.15.49:0/1729685014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 msgr2=0x55dc0bac8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 0x55dc0bac8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ed56a640 1 -- 172.21.15.49:0/1729685014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0b8087e0 con 0x55dc0bac9710 2024-03-16T08:24:39.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.565+0000 7f00ecd69640 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 0x55dc0bac8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.566+0000 7f00ed56a640 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f00d8011ad0 tx=0x7f00d8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3036a02263bca63a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.566+0000 7f00ed56a640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 0x7f00d00a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00d800f850 tx=0x7f00d801e000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.566+0000 7f00ceffd640 1 -- 172.21.15.49:0/1729685014 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8011050 con 0x55dc0bac9710 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.566+0000 7f00ceffd640 1 -- 172.21.15.49:0/1729685014 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d801ddc0 con 0x55dc0bac9710 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.566+0000 7f00ceffd640 1 -- 172.21.15.49:0/1729685014 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8010ad0 con 0x55dc0bac9710 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- 172.21.15.49:0/1729685014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 msgr2=0x55dc0bac8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f00d8011ad0 tx=0x7f00d8011f90 comp rx=0 tx=0).stop 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- 172.21.15.49:0/1729685014 shutdown_connections 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bad1da0 0x55dc0bad4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac8a30 0x55dc0bac8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/1729685014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bac8460 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- 172.21.15.49:0/1729685014 >> 172.21.15.49:0/1729685014 conn(0x55dc0ba8fc50 msgr2=0x55dc0bac3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- 172.21.15.49:0/1729685014 shutdown_connections 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- 172.21.15.49:0/1729685014 wait complete. 2024-03-16T08:24:39.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 -- start start 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.567+0000 7f00f1bb9740 1 Processor -- start 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0baced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bacf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc0bad4bb0 con 0x55dc0bac9710 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc0b9e80d0 con 0x55dc0bad1da0 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc0bad4d30 con 0x55dc0bac8a30 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 10 client.0 _ll_register_callbacks cb 0x55dc0b9b1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59664/0 (socket says 172.21.15.49:59664) 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 -- 172.21.15.49:0/2121138363 learned_addr learned my addr 172.21.15.49:0/2121138363 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0baced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00ed56a640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bacf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 -- 172.21.15.49:0/2121138363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 msgr2=0x55dc0baced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0baced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 -- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 msgr2=0x55dc0bacf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bacf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 -- 172.21.15.49:0/2121138363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0b868f70 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f00dc036040 tx=0x7f00dc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b9a6f1461a4eec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc004240 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f00dc0043e0 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.568+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d801ddc0 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f00dc02ccf0 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 10 client.17355 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc0b700170 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d801ddc0 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f00dc04f070 con 0x55dc0bac9710 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 10 client.17355 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:39.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 20 client.17355 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bae1e80 0x55dc0bae42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 4 client.17355 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0ba39610 con 0x55dc0bae1e80 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.569+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bae1e80 0x55dc0bae42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bae1e80 0x55dc0bae42c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f41ec32e5165cf server_cookie=7834a1202dabf871 in_seq=0 out_seq=0 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00c7fff640 10 client.17355 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00c7fff640 1 -- 172.21.15.49:0/2121138363 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dc0ba39610 con 0x55dc0bae1e80 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00c7fff640 10 client.17355 handle_command_reply: tid=0 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00f1bb9740 1 client.17355 shutdown 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 msgr2=0x55dc0bacf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.570+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f00dc036040 tx=0x7f00dc005d50 comp rx=0 tx=0).stop 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 shutdown_connections 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bae1e80 0x55dc0bae42c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bacf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bacf250 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00c7fff640 0 client.17355 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2121138363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0baced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 >> 172.21.15.49:0/2121138363 conn(0x55dc0ba8fc50 msgr2=0x55dc0bac51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 shutdown_connections 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2121138363 wait complete. 2024-03-16T08:24:39.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.571+0000 7f00f1bb9740 20 client.17355 trim_cache size 0 max 16384 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 Processor -- start 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 -- start start 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bad1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 0x55dc0badd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81afd60 con 0x55dc0bad1da0 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ad630 con 0x55dc0bac9710 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.588+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ab4a0 con 0x55dc0bac8a30 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ecd69640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bad1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 0x55dc0badd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59670/0 (socket says 172.21.15.49:59670) 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 -- 172.21.15.49:0/3995995737 learned_addr learned my addr 172.21.15.49:0/3995995737 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 -- 172.21.15.49:0/3995995737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 msgr2=0x55dc0bad1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bad1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 -- 172.21.15.49:0/3995995737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 msgr2=0x55dc0badd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 0x55dc0badd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 -- 172.21.15.49:0/3995995737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0bac7a60 con 0x55dc0bad1da0 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ecd69640 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bad1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00e7fff640 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 0x55dc0badd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ed56a640 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00d8010720 tx=0x7f00d80117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08e9539ff96930f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ce7fc640 1 -- 172.21.15.49:0/3995995737 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8034070 con 0x55dc0bad1da0 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ce7fc640 1 -- 172.21.15.49:0/3995995737 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d801ddc0 con 0x55dc0bad1da0 2024-03-16T08:24:39.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.589+0000 7f00ce7fc640 1 -- 172.21.15.49:0/3995995737 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8005330 con 0x55dc0bad1da0 2024-03-16T08:24:39.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- 172.21.15.49:0/3995995737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 msgr2=0x55dc0bae9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00d8010720 tx=0x7f00d80117c0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- 172.21.15.49:0/3995995737 shutdown_connections 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bad1da0 0x55dc0bae9b90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bac9710 0x55dc0badd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/3995995737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bad1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- 172.21.15.49:0/3995995737 >> 172.21.15.49:0/3995995737 conn(0x55dc0ba8fc50 msgr2=0x55dc0bac4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- 172.21.15.49:0/3995995737 shutdown_connections 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- 172.21.15.49:0/3995995737 wait complete. 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 -- start start 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.590+0000 7f00f1bb9740 1 Processor -- start 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bada1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bad7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81abaf0 con 0x55dc0bac9710 2024-03-16T08:24:39.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81acf60 con 0x55dc0bad1da0 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ac1c0 con 0x55dc0bac8a30 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00f1bb9740 10 client.0 _ll_register_callbacks cb 0x7f00dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59672/0 (socket says 172.21.15.49:59672) 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 -- 172.21.15.49:0/2541037979 learned_addr learned my addr 172.21.15.49:0/2541037979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00ed56a640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bad7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bada1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 -- 172.21.15.49:0/2541037979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 msgr2=0x55dc0bada1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bada1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 -- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 msgr2=0x55dc0bad7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bad7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 -- 172.21.15.49:0/2541037979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0bad7b70 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00e7fff640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f00dc005530 tx=0x7f00dc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3969f80a528fd0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.591+0000 7f00c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc055040 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7f00dc0035f0 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d801ddc0 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f00dc02cd90 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 10 client.17361 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc0bb490f0 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d801ddc0 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f00dc061070 con 0x55dc0bac9710 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 10 client.17361 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:39.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 20 client.17361 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bb498f0 0x55dc0bb4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 4 client.17361 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc0ba33960 con 0x55dc0bb498f0 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.592+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bb498f0 0x55dc0bb4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.593+0000 7f00c7fff640 10 client.17361 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.593+0000 7f00ecd69640 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bb498f0 0x55dc0bb4bd10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723c460c5d466289 server_cookie=2bb91115573da0dd in_seq=0 out_seq=0 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00c7fff640 1 -- 172.21.15.49:0/2541037979 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc0ba33960 con 0x55dc0bb498f0 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00c7fff640 10 client.17361 handle_command_reply: tid=0 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 client.17361 shutdown 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 msgr2=0x55dc0bad6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f00dc005530 tx=0x7f00dc004610 comp rx=0 tx=0).stop 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 shutdown_connections 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc0bb498f0 0x55dc0bb4bd10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0bad1da0 0x55dc0bad7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0bac9710 0x55dc0bad6fc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 --2- 172.21.15.49:0/2541037979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc0bac8a30 0x55dc0bada1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00c7fff640 0 client.17361 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 >> 172.21.15.49:0/2541037979 conn(0x55dc0ba8fc50 msgr2=0x55dc0bacdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 shutdown_connections 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 1 -- 172.21.15.49:0/2541037979 wait complete. 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.594+0000 7f00f1bb9740 20 client.17361 trim_cache size 0 max 16384 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "ce86367a-f19b-4df8-8f52-e6e368642bd4", 2024-03-16T08:24:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:24:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:39.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 msgr2=0x7f00d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 0x7f00d00a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00d800f850 tx=0x7f00d801e000 comp rx=0 tx=0).stop 2024-03-16T08:24:39.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 msgr2=0x7f00e8198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eb90 tx=0x7f00e000c6a0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 shutdown_connections 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f00d00a2970 0x7f00d00a4e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00e8107790 0x7f00e819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e8102140 0x7f00e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 --2- 172.21.15.49:0/4265473200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00e8101780 0x7f00e8198100 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 >> 172.21.15.49:0/4265473200 conn(0x7f00e80fd210 msgr2=0x7f00e80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 shutdown_connections 2024-03-16T08:24:39.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.595+0000 7f00eefcd640 1 -- 172.21.15.49:0/4265473200 wait complete. 2024-03-16T08:24:39.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:24:39.607 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:24:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/3891303364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4109990 msgr2=0x7fa7b4109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/3891303364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4109990 0x7fa7b4109d70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009860 tx=0x7fa7a802bb70 comp rx=0 tx=0).stop 2024-03-16T08:24:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/3891303364 shutdown_connections 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/3891303364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4104340 0x7fa7b4110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/3891303364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b4103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/3891303364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4109990 0x7fa7b4109d70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/3891303364 >> 172.21.15.49:0/3891303364 conn(0x7fa7b40ff450 msgr2=0x7fa7b4101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/3891303364 shutdown_connections 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/3891303364 wait complete. 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.716+0000 7fa7bc4f1640 1 Processor -- start 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 -- start start 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4104340 0x7fa7b419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4113da0 con 0x7fa7b4109990 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4113c20 con 0x7fa7b4104340 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7bc4f1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4113f20 con 0x7fa7b4103980 2024-03-16T08:24:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59684/0 (socket says 172.21.15.49:59684) 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 -- 172.21.15.49:0/1722664395 learned_addr learned my addr 172.21.15.49:0/1722664395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4104340 0x7fa7b419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 -- 172.21.15.49:0/1722664395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 msgr2=0x7fa7b419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 -- 172.21.15.49:0/1722664395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4104340 msgr2=0x7fa7b419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4104340 0x7fa7b419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 -- 172.21.15.49:0/1722664395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b419f2b0 con 0x7fa7b4109990 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.717+0000 7fa7baa8e640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009fd0 tx=0x7fa7ac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.718+0000 7fa7a77fe640 1 -- 172.21.15.49:0/1722664395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac004b20 con 0x7fa7b4109990 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.718+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b419f5a0 con 0x7fa7b4109990 2024-03-16T08:24:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.718+0000 7fa7a77fe640 1 -- 172.21.15.49:0/1722664395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac003e50 con 0x7fa7b4109990 2024-03-16T08:24:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.718+0000 7fa7a77fe640 1 -- 172.21.15.49:0/1722664395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00b630 con 0x7fa7b4109990 2024-03-16T08:24:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.718+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a6e30 con 0x7fa7b4109990 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.720+0000 7fa7a77fe640 1 -- 172.21.15.49:0/1722664395 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa7ac01d020 con 0x7fa7b4109990 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.720+0000 7fa7bf1b9740 1 Processor -- start 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.720+0000 7fa7bf1b9740 1 -- start start 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 0x561002112110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002112650 0x561002114a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56100206d500 con 0x561002109150 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561002015be0 con 0x561002112650 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5610020695a0 con 0x561002109a70 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7a77fe640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59686/0 (socket says 172.21.15.49:59686) 2024-03-16T08:24:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/3088940537 learned_addr learned my addr 172.21.15.49:0/3088940537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 0x561002112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002112650 0x561002114a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7baa8e640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7a77fe640 1 -- 172.21.15.49:0/1722664395 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7fa7ac0cee00 con 0x7fa7b4109990 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 -- 172.21.15.49:0/3088940537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002112650 msgr2=0x561002114a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002112650 0x561002114a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 -- 172.21.15.49:0/3088940537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 msgr2=0x561002109530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 -- 172.21.15.49:0/3088940537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610020294b0 con 0x561002109a70 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7baa8e640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac09b3b0 tx=0x7fa7ac0bb040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.721+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 0x561002112110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005940 tx=0x7fa7a80058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb428b7f2ebb452 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7a4bfe640 1 -- 172.21.15.49:0/3088940537 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8039040 con 0x561002109a70 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7a4bfe640 1 -- 172.21.15.49:0/3088940537 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8004230 con 0x561002109a70 2024-03-16T08:24:39.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7a4bfe640 1 -- 172.21.15.49:0/3088940537 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a802c970 con 0x561002109a70 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/3088940537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 msgr2=0x561002112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 0x561002112110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005940 tx=0x7fa7a80058d0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/3088940537 shutdown_connections 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002112650 0x561002114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109a70 0x561002112110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/3088940537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.722+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/3088940537 >> 172.21.15.49:0/3088940537 conn(0x561002103880 msgr2=0x561002103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/3088940537 shutdown_connections 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/3088940537 wait complete. 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- start start 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 Processor -- start 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109150 0x56100210f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x561002110100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5610021153e0 con 0x561002109a70 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561002029160 con 0x561002109150 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561002115560 con 0x561002112650 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7bf1b9740 10 client.0 _ll_register_callbacks cb 0x56100200d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7ba28d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7ba28d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59702/0 (socket says 172.21.15.49:59702) 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7ba28d640 1 -- 172.21.15.49:0/836499456 learned_addr learned my addr 172.21.15.49:0/836499456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109150 0x56100210f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.723+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x561002110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 -- 172.21.15.49:0/836499456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 msgr2=0x561002110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x561002110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 -- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109150 msgr2=0x56100210f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109150 0x56100210f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 -- 172.21.15.49:0/836499456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561001e4b7d0 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8037040 tx=0x7fa7a8034910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e3ab78ea9194c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8039040 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7fa7a802c6f0 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a8004230 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa7a8032400 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7bf1b9740 10 client.17373 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561001d813f0 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8004230 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.724+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa7a8057030 con 0x561002109a70 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 10 client.17373 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:39.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 20 client.17373 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002122470 0x561002124860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 4 client.17373 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56100207aa40 con 0x561002122470 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002122470 0x561002124860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002122470 0x561002124860 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dd828be1d6be1e server_cookie=6f79a48178598c86 in_seq=0 out_seq=0 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.725+0000 7fa79dffb640 10 client.17373 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa79dffb640 1 -- 172.21.15.49:0/836499456 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56100207aa40 con 0x561002122470 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa79dffb640 10 client.17373 handle_command_reply: tid=0 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 client.17373 shutdown 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 msgr2=0x56100210fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8037040 tx=0x7fa7a8034910 comp rx=0 tx=0).stop 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 shutdown_connections 2024-03-16T08:24:39.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002122470 0x561002124860 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x561002110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa79dffb640 0 client.17373 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109a70 0x56100210fbc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/836499456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109150 0x56100210f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 >> 172.21.15.49:0/836499456 conn(0x561002103880 msgr2=0x5610021055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 shutdown_connections 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/836499456 wait complete. 2024-03-16T08:24:39.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.726+0000 7fa7bf1b9740 20 client.17373 trim_cache size 0 max 16384 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 Processor -- start 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 -- start start 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x5610021088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x56100211e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41b1ed0 con 0x561002109150 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41af7a0 con 0x561002109a70 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41ad610 con 0x561002112650 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.743+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59706/0 (socket says 172.21.15.49:59706) 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/405495199 learned_addr learned my addr 172.21.15.49:0/405495199 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x5610021088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x56100211e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 -- 172.21.15.49:0/405495199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 msgr2=0x56100211e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x56100211e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 -- 172.21.15.49:0/405495199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 msgr2=0x561002111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 -- 172.21.15.49:0/405495199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100210e630 con 0x561002109a70 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x5610021088f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8032cf0 tx=0x7fa7a8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32167e7f7d3c8608 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa79ffff640 1 -- 172.21.15.49:0/405495199 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8039070 con 0x561002109a70 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa79ffff640 1 -- 172.21.15.49:0/405495199 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a802c9f0 con 0x561002109a70 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.744+0000 7fa79ffff640 1 -- 172.21.15.49:0/405495199 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8058650 con 0x561002109a70 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/405495199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 msgr2=0x5610021088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x5610021088f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8032cf0 tx=0x7fa7a8000bc0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/405495199 shutdown_connections 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002112650 0x56100211e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x5610021088f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/405495199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002109150 0x561002111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/405495199 >> 172.21.15.49:0/405495199 conn(0x561002103880 msgr2=0x561002102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/405495199 shutdown_connections 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/405495199 wait complete. 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- start start 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 Processor -- start 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109150 0x5610021167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x561002116d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41adc60 con 0x561002112650 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41af0d0 con 0x561002109a70 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41ae330 con 0x561002109150 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7bf1b9740 10 client.0 _ll_register_callbacks cb 0x7fa7b0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7b9a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59712/0 (socket says 172.21.15.49:59712) 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/332547049 learned_addr learned my addr 172.21.15.49:0/332547049 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.745+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x561002116d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109150 0x5610021167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/332547049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109150 msgr2=0x5610021167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109150 0x5610021167c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 msgr2=0x561002116d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x561002116d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 -- 172.21.15.49:0/332547049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561001d410f0 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x561002116d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7b9a8c640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0025de0 tx=0x7fa7b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0b94d55284b2d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0023040 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8492+0+0 (secure 0 0 0) 0x7fa7b00256e0 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a802c9f0 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa7b0004020 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7bf1b9740 10 client.17379 fetch_fsmap learned FSMap version 24 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5610020349d0 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.746+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a802c9f0 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa7b0038050 con 0x561002112650 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 10 client.17379 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:24:39.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 20 client.17379 populate_metadata read hostname 'smithi049' 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002189070 0x56100218b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 4 client.17379 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561002074d90 con 0x561002189070 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002189070 0x56100218b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa79dffb640 10 client.17379 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7ba28d640 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002189070 0x56100218b490 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6126be48eb1d98 server_cookie=673c55b6374e9abf in_seq=0 out_seq=0 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa79dffb640 1 -- 172.21.15.49:0/332547049 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561002074d90 con 0x561002189070 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa79dffb640 10 client.17379 handle_command_reply: tid=0 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 client.17379 shutdown 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 msgr2=0x561002187430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0025de0 tx=0x7fa7b0005f90 comp rx=0 tx=0).stop 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 shutdown_connections 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.747+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561002189070 0x56100218b490 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561002112650 0x561002187430 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561002109a70 0x561002116d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa79dffb640 0 client.17379 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 --2- 172.21.15.49:0/332547049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561002109150 0x5610021167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 >> 172.21.15.49:0/332547049 conn(0x561002103880 msgr2=0x56100210f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 shutdown_connections 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 1 -- 172.21.15.49:0/332547049 wait complete. 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bf1b9740 20 client.17379 trim_cache size 0 max 16384 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ce86367a-f19b-4df8-8f52-e6e368642bd4": { 2024-03-16T08:24:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "ce86367a-f19b-4df8-8f52-e6e368642bd4", 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:24:39.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 msgr2=0x7fa7a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac09b3b0 tx=0x7fa7ac0bb040 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 msgr2=0x7fa7b419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009fd0 tx=0x7fa7ac00c800 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 shutdown_connections 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa7a00a2970 0x7fa7a00a4e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b4109990 0x7fa7b419eb80 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7b4104340 0x7fa7b419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 --2- 172.21.15.49:0/1722664395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b4103980 0x7fa7b419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 >> 172.21.15.49:0/1722664395 conn(0x7fa7b40ff450 msgr2=0x7fa7b40ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 shutdown_connections 2024-03-16T08:24:39.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:24:39.748+0000 7fa7bc4f1640 1 -- 172.21.15.49:0/1722664395 wait complete. 2024-03-16T08:24:39.760 INFO:tasks.cephfs.filesystem:scrub status for tag:ce86367a-f19b-4df8-8f52-e6e368642bd4 - {'path': '/', 'tag': 'ce86367a-f19b-4df8-8f52-e6e368642bd4', 'options': 'recursive,force'} 2024-03-16T08:24:40.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:40 smithi129 ceph-mon[36561]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:40.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:40 smithi154 ceph-mon[36471]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:40 smithi049 ceph-mon[31942]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:24:41.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:41 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:24:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:41 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:41 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:24:41.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:41 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:41.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:41 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:24:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:41 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:41 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:24:41.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:41 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:41 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:24:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:41 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:41.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:41 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:24:41.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:41 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:24:42.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:42 smithi129 ceph-mon[36561]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:42 smithi154 ceph-mon[36471]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:42.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:42 smithi049 ceph-mon[31942]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:43.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:43 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:24:43.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:43 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:24:43.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:43 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:24:44.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:44 smithi129 ceph-mon[36561]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:44.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:44 smithi154 ceph-mon[36471]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:44 smithi049 ceph-mon[31942]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:46.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:46 smithi129 ceph-mon[36561]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:46.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:46 smithi154 ceph-mon[36471]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:46.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:46 smithi049 ceph-mon[31942]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:48.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:48 smithi129 ceph-mon[36561]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:48.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:48 smithi154 ceph-mon[36471]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:48.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:48 smithi049 ceph-mon[31942]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:50.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:50 smithi129 ceph-mon[36561]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:50.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:50 smithi154 ceph-mon[36471]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:50 smithi049 ceph-mon[31942]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:52.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:52 smithi129 ceph-mon[36561]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:52.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:52 smithi154 ceph-mon[36471]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:52.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:52 smithi049 ceph-mon[31942]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:54.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:54 smithi129 ceph-mon[36561]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:54.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:54 smithi154 ceph-mon[36471]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:54.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:54 smithi049 ceph-mon[31942]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:24:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:56 smithi129 ceph-mon[36561]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:56.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:56 smithi154 ceph-mon[36471]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:56.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:56 smithi049 ceph-mon[31942]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:24:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:24:58 smithi129 ceph-mon[36561]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:24:58 smithi154 ceph-mon[36471]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:24:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:24:58 smithi049 ceph-mon[31942]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:00 smithi129 ceph-mon[36561]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:00 smithi154 ceph-mon[36471]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:00 smithi049 ceph-mon[31942]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:02 smithi129 ceph-mon[36561]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:02 smithi154 ceph-mon[36471]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:02 smithi049 ceph-mon[31942]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:04 smithi129 ceph-mon[36561]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:04 smithi154 ceph-mon[36471]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:04 smithi049 ceph-mon[31942]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:06.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:06 smithi129 ceph-mon[36561]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:25:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:06 smithi154 ceph-mon[36471]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:25:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:06 smithi049 ceph-mon[31942]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:25:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:08 smithi129 ceph-mon[36561]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:25:08.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:08 smithi129 ceph-mon[36561]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T08:25:08.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:08 smithi154 ceph-mon[36471]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:25:08.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:08 smithi154 ceph-mon[36471]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T08:25:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:08 smithi049 ceph-mon[31942]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:25:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:08 smithi049 ceph-mon[31942]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T08:25:09.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:25:09.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 -- 172.21.15.49:0/3499274007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c10be20 msgr2=0x7f663c10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 --2- 172.21.15.49:0/3499274007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c10be20 0x7f663c10e290 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6634009ca0 tx=0x7f663402c180 comp rx=0 tx=0).stop 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 -- 172.21.15.49:0/3499274007 shutdown_connections 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 --2- 172.21.15.49:0/3499274007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c10be20 0x7f663c10e290 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 --2- 172.21.15.49:0/3499274007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c103370 0x7f663c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 --2- 172.21.15.49:0/3499274007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c102a50 0x7f663c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 -- 172.21.15.49:0/3499274007 >> 172.21.15.49:0/3499274007 conn(0x7f663c077840 msgr2=0x7f663c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 -- 172.21.15.49:0/3499274007 shutdown_connections 2024-03-16T08:25:09.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.883+0000 7f6641abf640 1 -- 172.21.15.49:0/3499274007 wait complete. 2024-03-16T08:25:09.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 Processor -- start 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 -- start start 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c102a50 0x7f663c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 0x7f663c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f663c111bf0 con 0x7f663c103370 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f663c111a70 con 0x7f663c10be20 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f6641abf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f663c111d70 con 0x7f663c102a50 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f663affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f663affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48836/0 (socket says 172.21.15.49:48836) 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f663affd640 1 -- 172.21.15.49:0/3577907780 learned_addr learned my addr 172.21.15.49:0/3577907780 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f663bfff640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 0x7f663c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.884+0000 7f663b7fe640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c102a50 0x7f663c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 -- 172.21.15.49:0/3577907780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c102a50 msgr2=0x7f663c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c102a50 0x7f663c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 -- 172.21.15.49:0/3577907780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 msgr2=0x7f663c101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 0x7f663c101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 -- 172.21.15.49:0/3577907780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c1a6530 con 0x7f663c103370 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663bfff640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 0x7f663c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f663affd640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9f0 tx=0x7f662c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f6638ff9640 1 -- 172.21.15.49:0/3577907780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00cd80 con 0x7f663c103370 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c1a6820 con 0x7f663c103370 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f6638ff9640 1 -- 172.21.15.49:0/3577907780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c005500 con 0x7f663c103370 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f6638ff9640 1 -- 172.21.15.49:0/3577907780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c010690 con 0x7f663c103370 2024-03-16T08:25:09.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.885+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c1a6d20 con 0x7f663c103370 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.887+0000 7f6638ff9640 1 -- 172.21.15.49:0/3577907780 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f662c0046c0 con 0x7f663c103370 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 Processor -- start 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 -- start start 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49848c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 0x55b498497a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b4983f0500 con 0x55b49848ca70 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b498398be0 con 0x55b49848c150 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6644781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b4983ec5a0 con 0x55b498495650 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49848c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48842/0 (socket says 172.21.15.49:48842) 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 -- 172.21.15.49:0/3283456605 learned_addr learned my addr 172.21.15.49:0/3283456605 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f6638ff9640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 0x7f66280a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663affd640 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 0x55b498497a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663bfff640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 0x7f66280a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 -- 172.21.15.49:0/3283456605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 msgr2=0x55b498497a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 0x55b498497a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 -- 172.21.15.49:0/3283456605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 msgr2=0x55b49848c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49848c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663b7fe640 1 -- 172.21.15.49:0/3283456605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4983ac4b0 con 0x55b49848ca70 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.888+0000 7f663affd640 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 0x55b498497a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:09.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f6638ff9640 1 -- 172.21.15.49:0/3577907780 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f662c0cf1e0 con 0x7f663c103370 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f663b7fe640 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6624009cf0 tx=0x7f6624011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d289baa990f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f663bfff640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 0x7f66280a4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6634004af0 tx=0x7f663402c180 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f6631ffb640 1 -- 172.21.15.49:0/3283456605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624004230 con 0x55b49848ca70 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f6631ffb640 1 -- 172.21.15.49:0/3283456605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240043d0 con 0x55b49848ca70 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.889+0000 7f6631ffb640 1 -- 172.21.15.49:0/3283456605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66240198c0 con 0x55b49848ca70 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- 172.21.15.49:0/3283456605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 msgr2=0x55b498495110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6624009cf0 tx=0x7f6624011810 comp rx=0 tx=0).stop 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- 172.21.15.49:0/3283456605 shutdown_connections 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b498495650 0x55b498497a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b49848ca70 0x55b498495110 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 --2- 172.21.15.49:0/3283456605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49848c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- 172.21.15.49:0/3283456605 >> 172.21.15.49:0/3283456605 conn(0x55b498486880 msgr2=0x55b498486c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- 172.21.15.49:0/3283456605 shutdown_connections 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- 172.21.15.49:0/3283456605 wait complete. 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 -- start start 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.890+0000 7f6644781740 1 Processor -- start 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 0x55b498492690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848ca70 0x55b498492bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b498498440 con 0x55b498495650 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b4983ac160 con 0x55b49848ca70 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b4984985c0 con 0x55b49848c150 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f6644781740 10 client.0 _ll_register_callbacks cb 0x55b498390150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48854/0 (socket says 172.21.15.49:48854) 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 -- 172.21.15.49:0/1433700846 learned_addr learned my addr 172.21.15.49:0/1433700846 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 0x55b498492690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663affd640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848ca70 0x55b498492bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 -- 172.21.15.49:0/1433700846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 msgr2=0x55b498492690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 0x55b498492690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 -- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848ca70 msgr2=0x55b498492bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848ca70 0x55b498492bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 -- 172.21.15.49:0/1433700846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4981ce7d0 con 0x55b498495650 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 0x55b498492690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f663bfff640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6634036000 tx=0x7f6634005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3718d65da2e08b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f661affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634039040 con 0x55b498495650 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.891+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f6634032df0 con 0x55b498495650 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66240043d0 con 0x55b498495650 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f663405c050 con 0x55b498495650 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 10 client.17391 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b4981043f0 con 0x55b498495650 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240043d0 con 0x55b498495650 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f663403f070 con 0x55b498495650 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 10 client.17391 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 20 client.17391 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b4984a56b0 0x55b4984a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 4 client.17391 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4983fda40 con 0x55b4984a56b0 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.892+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b4984a56b0 0x55b4984a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.893+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b4984a56b0 0x55b4984a7b30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5a1f0e99f20719 server_cookie=50a1835774e71719 in_seq=0 out_seq=0 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.893+0000 7f661affd640 10 client.17391 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f661affd640 1 -- 172.21.15.49:0/1433700846 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b4983fda40 con 0x55b4984a56b0 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f661affd640 10 client.17391 handle_command_reply: tid=0 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 client.17391 shutdown 2024-03-16T08:25:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 msgr2=0x55b498493110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6634036000 tx=0x7f6634005c50 comp rx=0 tx=0).stop 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 shutdown_connections 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b4984a56b0 0x55b4984a7b30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498493110 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848ca70 0x55b498492bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 --2- 172.21.15.49:0/1433700846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848c150 0x55b498492690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f661affd640 0 client.17391 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 >> 172.21.15.49:0/1433700846 conn(0x55b498486880 msgr2=0x55b4984885d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 shutdown_connections 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 1 -- 172.21.15.49:0/1433700846 wait complete. 2024-03-16T08:25:09.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.894+0000 7f6644781740 20 client.17391 trim_cache size 0 max 16384 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.911+0000 7f6644781740 1 Processor -- start 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.911+0000 7f6644781740 1 -- start start 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b4984a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498485780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1b1fd0 con 0x55b498495650 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1af8a0 con 0x55b49848c150 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f6644781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1ad710 con 0x55b49848ca70 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498485780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47042/0 (socket says 172.21.15.49:47042) 2024-03-16T08:25:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 -- 172.21.15.49:0/1970351460 learned_addr learned my addr 172.21.15.49:0/1970351460 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498485780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48862/0 (socket says 172.21.15.49:48862) 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663affd640 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b4984a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 -- 172.21.15.49:0/1970351460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 msgr2=0x55b4984a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b4984a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 -- 172.21.15.49:0/1970351460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 msgr2=0x55b498485780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498485780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 -- 172.21.15.49:0/1970351460 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4980c40f0 con 0x55b49848c150 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f663b7fe640 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6624005d50 tx=0x7f6624011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f4e747ff277cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.912+0000 7f66317fa640 1 -- 172.21.15.49:0/1970351460 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624031070 con 0x55b49848c150 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f66317fa640 1 -- 172.21.15.49:0/1970351460 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662402bc30 con 0x55b49848c150 2024-03-16T08:25:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f66317fa640 1 -- 172.21.15.49:0/1970351460 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624003850 con 0x55b49848c150 2024-03-16T08:25:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- 172.21.15.49:0/1970351460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 msgr2=0x55b498494dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6624005d50 tx=0x7f6624011810 comp rx=0 tx=0).stop 2024-03-16T08:25:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- 172.21.15.49:0/1970351460 shutdown_connections 2024-03-16T08:25:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498485780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b4984a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 --2- 172.21.15.49:0/1970351460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b498494dc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- 172.21.15.49:0/1970351460 >> 172.21.15.49:0/1970351460 conn(0x55b498486880 msgr2=0x55b498485070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- 172.21.15.49:0/1970351460 shutdown_connections 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- 172.21.15.49:0/1970351460 wait complete. 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.913+0000 7f6644781740 1 -- start start 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 Processor -- start 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49849d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b498498df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1add60 con 0x55b498495650 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1af1d0 con 0x55b49848c150 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1ae430 con 0x55b49848ca70 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f6644781740 10 client.0 _ll_register_callbacks cb 0x7f662c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49849d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48864/0 (socket says 172.21.15.49:48864) 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663bfff640 1 -- 172.21.15.49:0/985900252 learned_addr learned my addr 172.21.15.49:0/985900252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663affd640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b498498df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 -- 172.21.15.49:0/985900252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 msgr2=0x55b498498df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b498498df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 -- 172.21.15.49:0/985900252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 msgr2=0x55b498499330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663b7fe640 1 -- 172.21.15.49:0/985900252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4983b79d0 con 0x55b49848c150 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.914+0000 7f663bfff640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f663b7fe640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49849d890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6624009c40 tx=0x7f66240096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685ad7dd26ba302b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624031040 con 0x55b49848c150 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f66240035f0 con 0x55b49848c150 2024-03-16T08:25:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662402bc30 con 0x55b49848c150 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6624042050 con 0x55b49848c150 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 10 client.27278 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b49850b680 con 0x55b49848c150 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662402bc30 con 0x55b49848c150 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f662402b7f0 con 0x55b49848c150 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 10 client.27278 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 10 client.27278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 20 client.27278 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b49850bba0 0x55b49850df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 4 client.27278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.915+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4983f7d90 con 0x55b49850bba0 2024-03-16T08:25:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f663affd640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b49850bba0 0x55b49850df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f661affd640 10 client.27278 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f663affd640 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b49850bba0 0x55b49850df90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60abcdb0dbec71ec server_cookie=cf0fcf25f195b7de in_seq=0 out_seq=0 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f661affd640 1 -- 172.21.15.49:0/985900252 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b4983f7d90 con 0x55b49850bba0 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f661affd640 10 client.27278 handle_command_reply: tid=0 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 client.27278 shutdown 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 msgr2=0x55b49849d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49849d890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6624009c40 tx=0x7f66240096b0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 shutdown_connections 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55b49850bba0 0x55b49850df90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b498495650 0x55b498499330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b49848ca70 0x55b498498df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 --2- 172.21.15.49:0/985900252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b49848c150 0x55b49849d890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f661affd640 0 client.27278 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 >> 172.21.15.49:0/985900252 conn(0x55b498486880 msgr2=0x55b498485900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 shutdown_connections 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 1 -- 172.21.15.49:0/985900252 wait complete. 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.916+0000 7f6644781740 20 client.27278 trim_cache size 0 max 16384 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:25:09.924 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 msgr2=0x7f66280a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 0x7f66280a4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6634004af0 tx=0x7f663402c180 comp rx=0 tx=0).stop 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 msgr2=0x7f663c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9f0 tx=0x7f662c00edf0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 shutdown_connections 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f66280a29a0 0x7f66280a4e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f663c10be20 0x7f663c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f663c103370 0x7f663c100be0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 --2- 172.21.15.49:0/3577907780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c102a50 0x7f663c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 >> 172.21.15.49:0/3577907780 conn(0x7f663c077840 msgr2=0x7f663c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 shutdown_connections 2024-03-16T08:25:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:09.917+0000 7f6641abf640 1 -- 172.21.15.49:0/3577907780 wait complete. 2024-03-16T08:25:09.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:25:09.929 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:25:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- 172.21.15.49:0/2700985113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 msgr2=0x7f76d410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 --2- 172.21.15.49:0/2700985113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d410d570 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009cd0 tx=0x7f76cc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- 172.21.15.49:0/2700985113 shutdown_connections 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 --2- 172.21.15.49:0/2700985113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d410d570 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 --2- 172.21.15.49:0/2700985113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 --2- 172.21.15.49:0/2700985113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 0x7f76d4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- 172.21.15.49:0/2700985113 >> 172.21.15.49:0/2700985113 conn(0x7f76d4077780 msgr2=0x7f76d4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- 172.21.15.49:0/2700985113 shutdown_connections 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- 172.21.15.49:0/2700985113 wait complete. 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 Processor -- start 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.036+0000 7f76dc4de640 1 -- start start 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 0x7f76d419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111bc0 con 0x7f76d4101060 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111a40 con 0x7f76d41066b0 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111d40 con 0x7f76d41006a0 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48892/0 (socket says 172.21.15.49:48892) 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 -- 172.21.15.49:0/516887909 learned_addr learned my addr 172.21.15.49:0/516887909 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76da27a640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76daa7b640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 0x7f76d419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 -- 172.21.15.49:0/516887909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 msgr2=0x7f76d41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 -- 172.21.15.49:0/516887909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 msgr2=0x7f76d419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 0x7f76d419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 -- 172.21.15.49:0/516887909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d419d090 con 0x7f76d4101060 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76da27a640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:10.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76d9a79640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f76c800e9f0 tx=0x7f76c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c800cd80 con 0x7f76d4101060 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c8005500 con 0x7f76d4101060 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d419d380 con 0x7f76d4101060 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8010690 con 0x7f76d4101060 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.037+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a4c10 con 0x7f76d4101060 2024-03-16T08:25:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.038+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4068a20 con 0x7f76d4101060 2024-03-16T08:25:10.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.039+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f76c80046c0 con 0x7f76d4101060 2024-03-16T08:25:10.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.040+0000 7f76c77fe640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 0x7f76c00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.040+0000 7f76da27a640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 0x7f76c00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.041+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f76c80cf3f0 con 0x7f76d4101060 2024-03-16T08:25:10.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.041+0000 7f76da27a640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 0x7f76c00a4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76d00095f0 tx=0x7f76d0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.044+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76c8053740 con 0x7f76d4101060 2024-03-16T08:25:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.269+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d4108c90 con 0x7f76d4101060 2024-03-16T08:25:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.270+0000 7f76c77fe640 1 -- 172.21.15.49:0/516887909 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f76c80afe20 con 0x7f76d4101060 2024-03-16T08:25:10.272 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:25:10.272 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:25:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:25:10.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 msgr2=0x7f76c00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 0x7f76c00a4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76d00095f0 tx=0x7f76d0005f90 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 msgr2=0x7f76d41985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f76c800e9f0 tx=0x7f76c800edf0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 shutdown_connections 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76c00a29f0 0x7f76c00a4eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d41066b0 0x7f76d419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d4101060 0x7f76d41985e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 --2- 172.21.15.49:0/516887909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d41006a0 0x7f76d41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 >> 172.21.15.49:0/516887909 conn(0x7f76d4077780 msgr2=0x7f76d40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 shutdown_connections 2024-03-16T08:25:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.272+0000 7f76dc4de640 1 -- 172.21.15.49:0/516887909 wait complete. 2024-03-16T08:25:10.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:25:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:10 smithi129 ceph-mon[36561]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 -- 172.21.15.49:0/930187658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 msgr2=0x7f5588106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 --2- 172.21.15.49:0/930187658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f5588106af0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5584009cd0 tx=0x7f558402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 -- 172.21.15.49:0/930187658 shutdown_connections 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 --2- 172.21.15.49:0/930187658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55881010c0 0x7f558810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 --2- 172.21.15.49:0/930187658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5588100700 0x7f5588100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 --2- 172.21.15.49:0/930187658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f5588106af0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 -- 172.21.15.49:0/930187658 >> 172.21.15.49:0/930187658 conn(0x7f5588077840 msgr2=0x7f5588077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 -- 172.21.15.49:0/930187658 shutdown_connections 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 -- 172.21.15.49:0/930187658 wait complete. 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.394+0000 7f5590705640 1 Processor -- start 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 -- start start 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588100700 0x7f558819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55881010c0 0x7f558819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111c20 con 0x7f5588106710 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111aa0 con 0x7f5588100700 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f5590705640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111da0 con 0x7f55881010c0 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48900/0 (socket says 172.21.15.49:48900) 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 -- 172.21.15.49:0/3154396124 learned_addr learned my addr 172.21.15.49:0/3154396124 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558dca0640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55881010c0 0x7f558819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558e4a1640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588100700 0x7f558819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 -- 172.21.15.49:0/3154396124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55881010c0 msgr2=0x7f558819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55881010c0 0x7f558819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 -- 172.21.15.49:0/3154396124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588100700 msgr2=0x7f558819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588100700 0x7f558819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 -- 172.21.15.49:0/3154396124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558819f2a0 con 0x7f5588106710 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f558eca2640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f558000d950 tx=0x7f558000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.395+0000 7f557b7fe640 1 -- 172.21.15.49:0/3154396124 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580017030 con 0x7f5588106710 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.396+0000 7f557b7fe640 1 -- 172.21.15.49:0/3154396124 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5580004020 con 0x7f5588106710 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.396+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558819f590 con 0x7f5588106710 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.396+0000 7f557b7fe640 1 -- 172.21.15.49:0/3154396124 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580003c00 con 0x7f5588106710 2024-03-16T08:25:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.396+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881a6e20 con 0x7f5588106710 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f557b7fe640 1 -- 172.21.15.49:0/3154396124 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f558001a070 con 0x7f5588106710 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 Processor -- start 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 -- start start 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f81007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f557b7fe640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 0x7f55740a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5557f805b490 con 0x5557f80f8150 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5557f8003c90 con 0x5557f80f7830 2024-03-16T08:25:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.398+0000 7f55933c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557f8057530 con 0x5557f8100d30 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f81007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558eca2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47086/0 (socket says 172.21.15.49:47086) 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 -- 172.21.15.49:0/390902429 learned_addr learned my addr 172.21.15.49:0/390902429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 0x7f55740a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f557b7fe640 1 -- 172.21.15.49:0/3154396124 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f55800ce5c0 con 0x7f5588106710 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 -- 172.21.15.49:0/390902429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 msgr2=0x5557f8103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 -- 172.21.15.49:0/390902429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 msgr2=0x5557f80f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 -- 172.21.15.49:0/390902429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f80be500 con 0x5557f80f8150 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558eca2640 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558e4a1640 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f81007f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f558402c700 tx=0x7f5584009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d6dfaa7093ac72 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f558dca0640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 0x7f55740a4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f557c012a10 tx=0x7f557c00b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f5578ff9640 1 -- 172.21.15.49:0/390902429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584043070 con 0x5557f80f8150 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f5578ff9640 1 -- 172.21.15.49:0/390902429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584032c90 con 0x5557f80f8150 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.399+0000 7f5578ff9640 1 -- 172.21.15.49:0/390902429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584039040 con 0x5557f80f8150 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- 172.21.15.49:0/390902429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 msgr2=0x5557f81007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f81007f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f558402c700 tx=0x7f5584009900 comp rx=0 tx=0).stop 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- 172.21.15.49:0/390902429 shutdown_connections 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f81007f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 --2- 172.21.15.49:0/390902429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- 172.21.15.49:0/390902429 >> 172.21.15.49:0/390902429 conn(0x5557f80f1f60 msgr2=0x5557f80f2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- 172.21.15.49:0/390902429 shutdown_connections 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- 172.21.15.49:0/390902429 wait complete. 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.400+0000 7f55933c5740 1 -- start start 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 Processor -- start 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5557f8103ac0 con 0x5557f80f8150 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5557f8017230 con 0x5557f80f7830 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557f8103c40 con 0x5557f8100d30 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f55933c5740 10 client.0 _ll_register_callbacks cb 0x5557f8053010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48914/0 (socket says 172.21.15.49:48914) 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 -- 172.21.15.49:0/119745364 learned_addr learned my addr 172.21.15.49:0/119745364 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558e4a1640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558eca2640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 -- 172.21.15.49:0/119745364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 msgr2=0x5557f80fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 -- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 msgr2=0x5557f80fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558dca0640 1 -- 172.21.15.49:0/119745364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f8026100 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.401+0000 7f558e4a1640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f558dca0640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f557c0042c0 tx=0x7f557c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f40c9c31017af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c004470 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f557c005020 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584032c90 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f557c005220 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 10 client.17415 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5557f7d6f3f0 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584032c90 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f557c0035f0 con 0x5557f80f8150 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 10 client.17415 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 20 client.17415 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8110f20 0x5557f8113370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 4 client.17415 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.402+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f80689d0 con 0x5557f8110f20 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.403+0000 7f558e4a1640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8110f20 0x5557f8113370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.403+0000 7f5571ffb640 10 client.17415 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.403+0000 7f558e4a1640 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8110f20 0x5557f8113370 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed3dbc5c2b3f4c14 server_cookie=5f58b86af6057854 in_seq=0 out_seq=0 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f5571ffb640 1 -- 172.21.15.49:0/119745364 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5557f80689d0 con 0x5557f8110f20 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f5571ffb640 10 client.17415 handle_command_reply: tid=0 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 client.17415 shutdown 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 msgr2=0x5557f80fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f557c0042c0 tx=0x7f557c004730 comp rx=0 tx=0).stop 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 shutdown_connections 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8110f20 0x5557f8113370 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f5571ffb640 0 client.17415 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f80fe1c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 --2- 172.21.15.49:0/119745364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f80fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.404+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 >> 172.21.15.49:0/119745364 conn(0x5557f80f1f60 msgr2=0x5557f80f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.405+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 shutdown_connections 2024-03-16T08:25:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.405+0000 7f55933c5740 1 -- 172.21.15.49:0/119745364 wait complete. 2024-03-16T08:25:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.405+0000 7f55933c5740 20 client.17415 trim_cache size 0 max 16384 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 Processor -- start 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 -- start start 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f81003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b1f40 con 0x5557f80f8150 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55881af810 con 0x5557f80f7830 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f55933c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55881ad680 con 0x5557f8100d30 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f558dca0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f558dca0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48920/0 (socket says 172.21.15.49:48920) 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f558dca0640 1 -- 172.21.15.49:0/982733333 learned_addr learned my addr 172.21.15.49:0/982733333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f558e4a1640 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f81003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.418+0000 7f558eca2640 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 -- 172.21.15.49:0/982733333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 msgr2=0x5557f80fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 -- 172.21.15.49:0/982733333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 msgr2=0x5557f81003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f81003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 -- 172.21.15.49:0/982733333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f7e397d0 con 0x5557f80f8150 2024-03-16T08:25:10.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f558dca0640 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f557c009710 tx=0x7f557c005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22365afc3445e73c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f5573fff640 1 -- 172.21.15.49:0/982733333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00b9c0 con 0x5557f80f8150 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f5573fff640 1 -- 172.21.15.49:0/982733333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c0035f0 con 0x5557f80f8150 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.419+0000 7f5573fff640 1 -- 172.21.15.49:0/982733333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c004a00 con 0x5557f80f8150 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- 172.21.15.49:0/982733333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 msgr2=0x5557f810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f557c009710 tx=0x7f557c005a70 comp rx=0 tx=0).stop 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- 172.21.15.49:0/982733333 shutdown_connections 2024-03-16T08:25:10.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f80fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f8150 0x5557f810c860 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- 172.21.15.49:0/982733333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f7830 0x5557f81003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- 172.21.15.49:0/982733333 >> 172.21.15.49:0/982733333 conn(0x5557f80f1f60 msgr2=0x5557f80fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- 172.21.15.49:0/982733333 shutdown_connections 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- 172.21.15.49:0/982733333 wait complete. 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- start start 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 Processor -- start 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f8150 0x5557f816fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55881adcd0 con 0x5557f80f7830 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55881af140 con 0x5557f80f8150 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55881ae3a0 con 0x5557f8100d30 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f55933c5740 10 client.0 _ll_register_callbacks cb 0x7f557c012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f558e4a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f558e4a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48930/0 (socket says 172.21.15.49:48930) 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f558e4a1640 1 -- 172.21.15.49:0/814291550 learned_addr learned my addr 172.21.15.49:0/814291550 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f558dca0640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f8150 0x5557f816fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.420+0000 7f558eca2640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 -- 172.21.15.49:0/814291550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 msgr2=0x5557f8103f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 -- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f8150 msgr2=0x5557f816fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f8150 0x5557f816fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 -- 172.21.15.49:0/814291550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f8104640 con 0x5557f80f7830 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f558e4a1640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f558402f020 tx=0x7f5584000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95e4169d0bbf891 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584039040 con 0x5557f80f7830 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f55840328a0 con 0x5557f80f7830 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f557c0035f0 con 0x5557f80f7830 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f558405f050 con 0x5557f80f7830 2024-03-16T08:25:10.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f55933c5740 10 client.17421 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5557f8176d10 con 0x5557f80f7830 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c0035f0 con 0x5557f80f7830 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.421+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f55840503b0 con 0x5557f80f7830 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 10 client.17421 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 20 client.17421 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8177070 0x5557f8179460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 4 client.17421 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557f8062d20 con 0x5557f8177070 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f558dca0640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8177070 0x5557f8179460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f558dca0640 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8177070 0x5557f8179460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d95e408f104b45ca server_cookie=ac13ec4a6faf2abc in_seq=0 out_seq=0 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f5571ffb640 10 client.17421 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f5571ffb640 1 -- 172.21.15.49:0/814291550 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557f8062d20 con 0x5557f8177070 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f5571ffb640 10 client.17421 handle_command_reply: tid=0 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 client.17421 shutdown 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 msgr2=0x5557f816f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f558402f020 tx=0x7f5584000f40 comp rx=0 tx=0).stop 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 shutdown_connections 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x5557f8177070 0x5557f8179460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557f8100d30 0x5557f8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557f80f8150 0x5557f816fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 --2- 172.21.15.49:0/814291550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557f80f7830 0x5557f816f570 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f5571ffb640 0 client.17421 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.422+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 >> 172.21.15.49:0/814291550 conn(0x5557f80f1f60 msgr2=0x5557f80f0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 shutdown_connections 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f55933c5740 1 -- 172.21.15.49:0/814291550 wait complete. 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f55933c5740 20 client.17421 trim_cache size 0 max 16384 2024-03-16T08:25:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 msgr2=0x7f55740a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 0x7f55740a4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f557c012a10 tx=0x7f557c00b390 comp rx=0 tx=0).stop 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 msgr2=0x7f558819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f558000d950 tx=0x7f558000de10 comp rx=0 tx=0).stop 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 shutdown_connections 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55740a29f0 0x7f55740a4eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5588106710 0x7f558819eb70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55881010c0 0x7f558819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 --2- 172.21.15.49:0/3154396124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588100700 0x7f558819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 >> 172.21.15.49:0/3154396124 conn(0x7f5588077840 msgr2=0x7f558810b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 shutdown_connections 2024-03-16T08:25:10.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.423+0000 7f5590705640 1 -- 172.21.15.49:0/3154396124 wait complete. 2024-03-16T08:25:10.434 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:10 smithi049 ceph-mon[31942]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:10.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:25:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:10 smithi154 ceph-mon[36471]: pgmap v244: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.543+0000 7f4d4737e640 1 -- 172.21.15.49:0/3152514189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400689c0 msgr2=0x7f4d40068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.543+0000 7f4d4737e640 1 --2- 172.21.15.49:0/3152514189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400689c0 0x7f4d40068e40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009930 tx=0x7f4d3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- 172.21.15.49:0/3152514189 shutdown_connections 2024-03-16T08:25:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- 172.21.15.49:0/3152514189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d40069380 0x7f4d4010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- 172.21.15.49:0/3152514189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400689c0 0x7f4d40068e40 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- 172.21.15.49:0/3152514189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40107760 0x7f4d40107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- 172.21.15.49:0/3152514189 >> 172.21.15.49:0/3152514189 conn(0x7f4d400777e0 msgr2=0x7f4d40077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- 172.21.15.49:0/3152514189 shutdown_connections 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- 172.21.15.49:0/3152514189 wait complete. 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 Processor -- start 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- start start 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 0x7f4d401980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40069380 0x7f4d40198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40111b70 con 0x7f4d40107760 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401119f0 con 0x7f4d400689c0 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.544+0000 7f4d4737e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40111cf0 con 0x7f4d40069380 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48952/0 (socket says 172.21.15.49:48952) 2024-03-16T08:25:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4511a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 0x7f4d401980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 -- 172.21.15.49:0/1829482798 learned_addr learned my addr 172.21.15.49:0/1829482798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d44919640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40069380 0x7f4d40198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 -- 172.21.15.49:0/1829482798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40069380 msgr2=0x7f4d40198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40069380 0x7f4d40198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 -- 172.21.15.49:0/1829482798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 msgr2=0x7f4d401980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 0x7f4d401980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 -- 172.21.15.49:0/1829482798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4019d0c0 con 0x7f4d40107760 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4511a640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 0x7f4d401980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4591b640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800efd0 tx=0x7f4d3800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d367fc640 1 -- 172.21.15.49:0/1829482798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003b70 con 0x7f4d40107760 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d367fc640 1 -- 172.21.15.49:0/1829482798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d38003d10 con 0x7f4d40107760 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4019d3b0 con 0x7f4d40107760 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d367fc640 1 -- 172.21.15.49:0/1829482798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380109f0 con 0x7f4d40107760 2024-03-16T08:25:10.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.545+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401a4c40 con 0x7f4d40107760 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.548+0000 7f4d367fc640 1 -- 172.21.15.49:0/1829482798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4d38004020 con 0x7f4d40107760 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.548+0000 7f4d49f6e740 1 Processor -- start 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.548+0000 7f4d49f6e740 1 -- start start 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b6509c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6512d30 0x5624b6515120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624b646d490 con 0x5624b650a150 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5624b6415c90 con 0x5624b6509830 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5624b6469530 con 0x5624b6512d30 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48958/0 (socket says 172.21.15.49:48958) 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 -- 172.21.15.49:0/1694967421 learned_addr learned my addr 172.21.15.49:0/1694967421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d44919640 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b6509c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d44919640 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6512d30 0x5624b6515120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d367fc640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 0x7f4d2c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 -- 172.21.15.49:0/1694967421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6512d30 msgr2=0x5624b6515120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6512d30 0x5624b6515120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 -- 172.21.15.49:0/1694967421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 msgr2=0x5624b6509c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b6509c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 -- 172.21.15.49:0/1694967421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b64d0500 con 0x5624b650a150 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4591b640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 0x7f4d2c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d367fc640 1 -- 172.21.15.49:0/1829482798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f4d380cf280 con 0x7f4d40107760 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.549+0000 7f4d4511a640 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0114f0 tx=0x7f4d3c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa3d794bcb1686e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d2b7fe640 1 -- 172.21.15.49:0/1694967421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c01e070 con 0x5624b650a150 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d2b7fe640 1 -- 172.21.15.49:0/1694967421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c011250 con 0x5624b650a150 2024-03-16T08:25:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d2b7fe640 1 -- 172.21.15.49:0/1694967421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c019990 con 0x5624b650a150 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 -- 172.21.15.49:0/1694967421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 msgr2=0x5624b65127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d4591b640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 0x7f4d2c0a4e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d380a3b40 tx=0x7f4d3804b040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0114f0 tx=0x7f4d3c0119b0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 -- 172.21.15.49:0/1694967421 shutdown_connections 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6512d30 0x5624b6515120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b65127f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/1694967421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b6509c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 -- 172.21.15.49:0/1694967421 >> 172.21.15.49:0/1694967421 conn(0x5624b6503f60 msgr2=0x5624b6504340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 -- 172.21.15.49:0/1694967421 shutdown_connections 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.550+0000 7f4d49f6e740 1 -- 172.21.15.49:0/1694967421 wait complete. 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 -- start start 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 Processor -- start 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b6510300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b6510840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624b6515b20 con 0x5624b6509830 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5624b6429230 con 0x5624b6512d30 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5624b6515ca0 con 0x5624b650a150 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d44919640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d49f6e740 10 client.0 _ll_register_callbacks cb 0x5624b6465010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d44919640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48968/0 (socket says 172.21.15.49:48968) 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d44919640 1 -- 172.21.15.49:0/2487597180 learned_addr learned my addr 172.21.15.49:0/2487597180 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b6510300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.551+0000 7f4d44919640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b6510840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 -- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 msgr2=0x5624b6510300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b6510300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 -- 172.21.15.49:0/2487597180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 msgr2=0x5624b6510840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b6510840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 -- 172.21.15.49:0/2487597180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b6438100 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d44919640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005c40 tx=0x7f4d3002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5033f1224369b457 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004490 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f4d300048d0 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c011250 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4d30004ad0 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d49f6e740 10 client.17433 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624b61813f0 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c011250 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.552+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4d300038c0 con 0x5624b6509830 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 10 client.17433 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 20 client.17433 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b6523060 0x5624b6525490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 4 client.17433 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b647a9d0 con 0x5624b6523060 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b6523060 0x5624b6525490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d28ff9640 10 client.17433 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.553+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b6523060 0x5624b6525490 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2932897dc4f7a455 server_cookie=b0c5c374cc322156 in_seq=0 out_seq=0 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2487597180 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5624b647a9d0 con 0x5624b6523060 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d28ff9640 10 client.17433 handle_command_reply: tid=0 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 client.17433 shutdown 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 msgr2=0x5624b650fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005c40 tx=0x7f4d3002bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 shutdown_connections 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b6523060 0x5624b6525490 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b6510840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b6510300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d28ff9640 0 client.17433 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2487597180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6509830 0x5624b650fdc0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 >> 172.21.15.49:0/2487597180 conn(0x5624b6503f60 msgr2=0x5624b6505cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 shutdown_connections 2024-03-16T08:25:10.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2487597180 wait complete. 2024-03-16T08:25:10.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.554+0000 7f4d49f6e740 20 client.17433 trim_cache size 0 max 16384 2024-03-16T08:25:10.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.567+0000 7f4d49f6e740 1 Processor -- start 2024-03-16T08:25:10.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 -- start start 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b65124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b651e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401afe40 con 0x5624b6512d30 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ad710 con 0x5624b6509830 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ab580 con 0x5624b650a150 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48984/0 (socket says 172.21.15.49:48984) 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 -- 172.21.15.49:0/2521458106 learned_addr learned my addr 172.21.15.49:0/2521458106 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b651e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b65124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 -- 172.21.15.49:0/2521458106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 msgr2=0x5624b651e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b651e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 -- 172.21.15.49:0/2521458106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 msgr2=0x5624b65124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b65124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 -- 172.21.15.49:0/2521458106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b624b7d0 con 0x5624b6512d30 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d44919640 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005ec0 tx=0x7f4d30005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe9648427ff9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.568+0000 7f4d2affd640 1 -- 172.21.15.49:0/2521458106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d300048d0 con 0x5624b6512d30 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d2affd640 1 -- 172.21.15.49:0/2521458106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d30004c70 con 0x5624b6512d30 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d2affd640 1 -- 172.21.15.49:0/2521458106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d300053e0 con 0x5624b6512d30 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2521458106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 msgr2=0x5624b6502e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005ec0 tx=0x7f4d30005b50 comp rx=0 tx=0).stop 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2521458106 shutdown_connections 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b6512d30 0x5624b6502e60 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b650a150 0x5624b651e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2521458106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6509830 0x5624b65124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2521458106 >> 172.21.15.49:0/2521458106 conn(0x5624b6503f60 msgr2=0x5624b6502750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2521458106 shutdown_connections 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2521458106 wait complete. 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 -- start start 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.569+0000 7f4d49f6e740 1 Processor -- start 2024-03-16T08:25:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6509830 0x5624b651aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b650c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401abbd0 con 0x5624b650a150 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ad040 con 0x5624b6512d30 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ac2a0 con 0x5624b6509830 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d49f6e740 10 client.0 _ll_register_callbacks cb 0x7f4d30009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48990/0 (socket says 172.21.15.49:48990) 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 -- 172.21.15.49:0/2162373740 learned_addr learned my addr 172.21.15.49:0/2162373740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d44919640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6509830 0x5624b651aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d44919640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b650c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 -- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6509830 msgr2=0x5624b651aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6509830 0x5624b651aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 -- 172.21.15.49:0/2162373740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 msgr2=0x5624b650c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b650c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.570+0000 7f4d4511a640 1 -- 172.21.15.49:0/2162373740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b650cee0 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d4511a640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c019540 tx=0x7f4d3c019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87ef22b8aa7979 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c01e040 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f4d3c009b50 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d30004c70 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c039050 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 10 client.17439 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624b65890c0 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d30004c70 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4d3c02b680 con 0x5624b650a150 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 10 client.17439 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 10 client.17439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 20 client.17439 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b65896d0 0x5624b658baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 4 client.17439 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624b6474d20 con 0x5624b65896d0 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.571+0000 7f4d44919640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b65896d0 0x5624b658baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d28ff9640 10 client.17439 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d44919640 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b65896d0 0x5624b658baf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a43956ef501223ba server_cookie=1808f99f0a54df44 in_seq=0 out_seq=0 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d28ff9640 1 -- 172.21.15.49:0/2162373740 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624b6474d20 con 0x5624b65896d0 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d28ff9640 10 client.17439 handle_command_reply: tid=0 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 client.17439 shutdown 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 msgr2=0x5624b650c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c019540 tx=0x7f4d3c019570 comp rx=0 tx=0).stop 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 shutdown_connections 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x5624b65896d0 0x5624b658baf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624b6512d30 0x5624b650c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d28ff9640 0 client.17439 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:10.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624b650a150 0x5624b650c460 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 --2- 172.21.15.49:0/2162373740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b6509830 0x5624b651aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 >> 172.21.15.49:0/2162373740 conn(0x5624b6503f60 msgr2=0x5624b6502fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 shutdown_connections 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 1 -- 172.21.15.49:0/2162373740 wait complete. 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.572+0000 7f4d49f6e740 20 client.17439 trim_cache size 0 max 16384 2024-03-16T08:25:10.580 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 msgr2=0x7f4d2c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 0x7f4d2c0a4e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d380a3b40 tx=0x7f4d3804b040 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 msgr2=0x7f4d4019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800efd0 tx=0x7f4d3800c6a0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 shutdown_connections 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4d2c0a2970 0x7f4d2c0a4e30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40107760 0x7f4d4019c990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d40069380 0x7f4d40198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 --2- 172.21.15.49:0/1829482798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d400689c0 0x7f4d401980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 >> 172.21.15.49:0/1829482798 conn(0x7f4d400777e0 msgr2=0x7f4d400fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 shutdown_connections 2024-03-16T08:25:10.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.573+0000 7f4d4737e640 1 -- 172.21.15.49:0/1829482798 wait complete. 2024-03-16T08:25:10.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:25:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/3610799751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc8069380 msgr2=0x7ffbc810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/3610799751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc8069380 0x7ffbc810e620 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009cd0 tx=0x7ffbb802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/3610799751 shutdown_connections 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/3610799751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc8069380 0x7ffbc810e620 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/3610799751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc80689c0 0x7ffbc8068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/3610799751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 0x7ffbc8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/3610799751 >> 172.21.15.49:0/3610799751 conn(0x7ffbc80777e0 msgr2=0x7ffbc8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/3610799751 shutdown_connections 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/3610799751 wait complete. 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 Processor -- start 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.695+0000 7ffbcfbcd640 1 -- start start 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc8069380 0x7ffbc8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 0x7ffbc819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111b70 con 0x7ffbc80689c0 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81119f0 con 0x7ffbc8107760 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111cf0 con 0x7ffbc8069380 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49010/0 (socket says 172.21.15.49:49010) 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 -- 172.21.15.49:0/2558207459 learned_addr learned my addr 172.21.15.49:0/2558207459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd168640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc8069380 0x7ffbc8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbce16a640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 0x7ffbc819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 -- 172.21.15.49:0/2558207459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc8069380 msgr2=0x7ffbc8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc8069380 0x7ffbc8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 -- 172.21.15.49:0/2558207459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 msgr2=0x7ffbc819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 0x7ffbc819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 -- 172.21.15.49:0/2558207459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc819d070 con 0x7ffbc80689c0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcd969640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000ea60 tx=0x7ffbc000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbc6ffd640 1 -- 172.21.15.49:0/2558207459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000cde0 con 0x7ffbc80689c0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbc6ffd640 1 -- 172.21.15.49:0/2558207459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc00035f0 con 0x7ffbc80689c0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc819d360 con 0x7ffbc80689c0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.696+0000 7ffbc6ffd640 1 -- 172.21.15.49:0/2558207459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0010670 con 0x7ffbc80689c0 2024-03-16T08:25:10.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.697+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81a4bf0 con 0x7ffbc80689c0 2024-03-16T08:25:10.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.699+0000 7ffbc6ffd640 1 -- 172.21.15.49:0/2558207459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ffbc00037a0 con 0x7ffbc80689c0 2024-03-16T08:25:10.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.699+0000 7ffbd27b9740 1 Processor -- start 2024-03-16T08:25:10.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.699+0000 7ffbd27b9740 1 -- start start 2024-03-16T08:25:10.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b47c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b48150 0x558e69b507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558e69aab490 con 0x558e69b48150 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e69a53c90 con 0x558e69b47830 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558e69aa7530 con 0x558e69b50d30 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbc6ffd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 0x7ffbb00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b48150 0x558e69b507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbce16a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b47c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36916/0 (socket says 172.21.15.49:36916) 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbce16a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b47c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47218/0 (socket says 172.21.15.49:47218) 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 -- 172.21.15.49:0/3444816418 learned_addr learned my addr 172.21.15.49:0/3444816418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbce16a640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 0x7ffbb00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbc6ffd640 1 -- 172.21.15.49:0/2558207459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7ffbc00cf090 con 0x7ffbc80689c0 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 -- 172.21.15.49:0/3444816418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 msgr2=0x558e69b47c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b47c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 -- 172.21.15.49:0/3444816418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b48150 msgr2=0x558e69b507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b48150 0x558e69b507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 -- 172.21.15.49:0/3444816418 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e69b0e500 con 0x558e69b50d30 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbce16a640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 0x7ffbb00a4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8005cc0 tx=0x7ffbb8005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.700+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009ca0 tx=0x7ffbbc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57dc47b39e5ef51b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbaffff640 1 -- 172.21.15.49:0/3444816418 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc01e070 con 0x558e69b50d30 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbaffff640 1 -- 172.21.15.49:0/3444816418 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbbc019b10 con 0x558e69b50d30 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbaffff640 1 -- 172.21.15.49:0/3444816418 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00e9c0 con 0x558e69b50d30 2024-03-16T08:25:10.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3444816418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 msgr2=0x558e69b53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009ca0 tx=0x7ffbbc011810 comp rx=0 tx=0).stop 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3444816418 shutdown_connections 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b53120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b48150 0x558e69b507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3444816418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b47c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3444816418 >> 172.21.15.49:0/3444816418 conn(0x558e69b41f60 msgr2=0x558e69b42340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3444816418 shutdown_connections 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.701+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3444816418 wait complete. 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 -- start start 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 Processor -- start 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b48150 0x558e69b4e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558e69b53b20 con 0x558e69b50d30 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e69a67230 con 0x558e69b47830 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558e69b53ca0 con 0x558e69b48150 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbd27b9740 10 client.0 _ll_register_callbacks cb 0x558e69aa3010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbce16a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.702+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47220/0 (socket says 172.21.15.49:47220) 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbce16a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49024/0 (socket says 172.21.15.49:49024) 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 -- 172.21.15.49:0/3013368333 learned_addr learned my addr 172.21.15.49:0/3013368333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b48150 0x558e69b4e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 -- 172.21.15.49:0/3013368333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b48150 msgr2=0x558e69b4e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b48150 0x558e69b4e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 -- 172.21.15.49:0/3013368333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 msgr2=0x558e69b4e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 -- 172.21.15.49:0/3013368333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e69a76100 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbcd969640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0097b40 tx=0x7ffbc009b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f80b2b3f6a625ff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc009bde0 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7ffbc00cca50 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc019b10 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffbc009b6e0 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbd27b9740 10 client.27332 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e697bf3f0 con 0x558e69b47830 2024-03-16T08:25:10.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.703+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbbc019b10 con 0x558e69b47830 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffbc00d1110 con 0x558e69b47830 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 10 client.27332 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 10 client.27332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 20 client.27332 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69b60fa0 0x558e69b63390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 4 client.27332 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e69ab89d0 con 0x558e69b60fa0 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69b60fa0 0x558e69b63390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbad7fa640 10 client.27332 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.704+0000 7ffbcd168640 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69b60fa0 0x558e69b63390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6b2dc245661f80 server_cookie=34aac3cc9b8ef7ef in_seq=0 out_seq=0 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbad7fa640 1 -- 172.21.15.49:0/3013368333 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558e69ab89d0 con 0x558e69b60fa0 2024-03-16T08:25:10.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbad7fa640 10 client.27332 handle_command_reply: tid=0 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 client.27332 shutdown 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 msgr2=0x558e69b4dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0097b40 tx=0x7ffbc009b5e0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 shutdown_connections 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69b60fa0 0x558e69b63390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b48150 0x558e69b4e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/3013368333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b47830 0x558e69b4dcb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbad7fa640 0 client.27332 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 >> 172.21.15.49:0/3013368333 conn(0x558e69b41f60 msgr2=0x558e69b43cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 shutdown_connections 2024-03-16T08:25:10.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 1 -- 172.21.15.49:0/3013368333 wait complete. 2024-03-16T08:25:10.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.705+0000 7ffbd27b9740 20 client.27332 trim_cache size 0 max 16384 2024-03-16T08:25:10.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 Processor -- start 2024-03-16T08:25:10.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 -- start start 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b5c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81afdb0 con 0x558e69b47830 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81ad680 con 0x558e69b48150 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81ab4f0 con 0x558e69b50d30 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49034/0 (socket says 172.21.15.49:49034) 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 -- 172.21.15.49:0/126894864 learned_addr learned my addr 172.21.15.49:0/126894864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd168640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbce16a640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b5c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 -- 172.21.15.49:0/126894864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 msgr2=0x558e69b5c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b5c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 -- 172.21.15.49:0/126894864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 msgr2=0x558e69b592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 -- 172.21.15.49:0/126894864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e698897d0 con 0x558e69b47830 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbce16a640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b5c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbcd969640 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00980b0 tx=0x7ffbc0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd2fe5703277c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbaf7fe640 1 -- 172.21.15.49:0/126894864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc009b040 con 0x558e69b47830 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.719+0000 7ffbaf7fe640 1 -- 172.21.15.49:0/126894864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc009bde0 con 0x558e69b47830 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbaf7fe640 1 -- 172.21.15.49:0/126894864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00cca50 con 0x558e69b47830 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- 172.21.15.49:0/126894864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 msgr2=0x558e69b503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00980b0 tx=0x7ffbc0020000 comp rx=0 tx=0).stop 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- 172.21.15.49:0/126894864 shutdown_connections 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b50d30 0x558e69b5c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/126894864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b47830 0x558e69b503e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- 172.21.15.49:0/126894864 >> 172.21.15.49:0/126894864 conn(0x558e69b41f60 msgr2=0x558e69b40750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- 172.21.15.49:0/126894864 shutdown_connections 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- 172.21.15.49:0/126894864 wait complete. 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 -- start start 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.720+0000 7ffbd27b9740 1 Processor -- start 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b47830 0x558e69b49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81abb40 con 0x558e69b50d30 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81acfb0 con 0x558e69b48150 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81ac210 con 0x558e69b47830 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbd27b9740 10 client.0 _ll_register_callbacks cb 0x7ffbc0003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbce16a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47236/0 (socket says 172.21.15.49:47236) 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 -- 172.21.15.49:0/1711903156 learned_addr learned my addr 172.21.15.49:0/1711903156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd969640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b47830 0x558e69b49f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 -- 172.21.15.49:0/1711903156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b47830 msgr2=0x558e69b49f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b47830 0x558e69b49f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 -- 172.21.15.49:0/1711903156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 msgr2=0x558e69b4ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.721+0000 7ffbcd168640 1 -- 172.21.15.49:0/1711903156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e69b4b3b0 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbcd168640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009a30 tx=0x7ffbbc009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df07261320b623c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc033060 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc009bde0 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7ffbbc005270 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc01e040 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 10 client.27338 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e69bcba10 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc009bde0 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffbbc0035f0 con 0x558e69b48150 2024-03-16T08:25:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 10 client.27338 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 10 client.27338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 20 client.27338 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69bcbfe0 0x558e69bce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 4 client.27338 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e69ab2d20 con 0x558e69bcbfe0 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.722+0000 7ffbcd969640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69bcbfe0 0x558e69bce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbad7fa640 10 client.27338 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbcd969640 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69bcbfe0 0x558e69bce400 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd0b6ccb7e2de2a server_cookie=5000314c39497ca0 in_seq=0 out_seq=0 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbad7fa640 1 -- 172.21.15.49:0/1711903156 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e69ab2d20 con 0x558e69bcbfe0 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbad7fa640 10 client.27338 handle_command_reply: tid=0 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 client.27338 shutdown 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 msgr2=0x558e69b4a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009a30 tx=0x7ffbbc009680 comp rx=0 tx=0).stop 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 shutdown_connections 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558e69bcbfe0 0x558e69bce400 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e69b50d30 0x558e69b4ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e69b48150 0x558e69b4a460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 --2- 172.21.15.49:0/1711903156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e69b47830 0x558e69b49f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbad7fa640 0 client.27338 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 >> 172.21.15.49:0/1711903156 conn(0x558e69b41f60 msgr2=0x558e69b40fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 shutdown_connections 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 1 -- 172.21.15.49:0/1711903156 wait complete. 2024-03-16T08:25:10.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.723+0000 7ffbd27b9740 20 client.27338 trim_cache size 0 max 16384 2024-03-16T08:25:10.733 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 msgr2=0x7ffbb00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 0x7ffbb00a4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8005cc0 tx=0x7ffbb8005c50 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 msgr2=0x7ffbc81980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000ea60 tx=0x7ffbc000ef20 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 shutdown_connections 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7ffbb00a29f0 0x7ffbb00a4eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbc8107760 0x7ffbc819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbc8069380 0x7ffbc8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 --2- 172.21.15.49:0/2558207459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbc80689c0 0x7ffbc81980d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 >> 172.21.15.49:0/2558207459 conn(0x7ffbc80777e0 msgr2=0x7ffbc80fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:10.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 shutdown_connections 2024-03-16T08:25:10.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:10.724+0000 7ffbcfbcd640 1 -- 172.21.15.49:0/2558207459 wait complete. 2024-03-16T08:25:11.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:11 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/516887909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:11.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:11 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/516887909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:11.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:11 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/516887909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:11.736 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:25:11.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:25:11.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 -- 172.21.15.49:0/2658456206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8100700 msgr2=0x7f6ce8100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 --2- 172.21.15.49:0/2658456206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8100700 0x7f6ce8100b80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009930 tx=0x7f6cd802bc20 comp rx=0 tx=0).stop 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 -- 172.21.15.49:0/2658456206 shutdown_connections 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 --2- 172.21.15.49:0/2658456206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 --2- 172.21.15.49:0/2658456206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8100700 0x7f6ce8100b80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 --2- 172.21.15.49:0/2658456206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8106710 0x7f6ce8106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 -- 172.21.15.49:0/2658456206 >> 172.21.15.49:0/2658456206 conn(0x7f6ce8077840 msgr2=0x7f6ce8077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 -- 172.21.15.49:0/2658456206 shutdown_connections 2024-03-16T08:25:11.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.860+0000 7f6cf0559640 1 -- 172.21.15.49:0/2658456206 wait complete. 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 Processor -- start 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 -- start start 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8100700 0x7f6ce8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 0x7f6ce819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8110b20 con 0x7f6ce8106710 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81109a0 con 0x7f6ce81010c0 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cf0559640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8110ca0 con 0x7f6ce8100700 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6ceeaf6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 0x7f6ce819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cee2f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8100700 0x7f6ce8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.861+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:47246/0 (socket says 172.21.15.49:47246) 2024-03-16T08:25:11.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cedaf4640 1 -- 172.21.15.49:0/1641444346 learned_addr learned my addr 172.21.15.49:0/1641444346 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 -- 172.21.15.49:0/1641444346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8100700 msgr2=0x7f6ce8197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8100700 0x7f6ce8197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 -- 172.21.15.49:0/1641444346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 msgr2=0x7f6ce8197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 -- 172.21.15.49:0/1641444346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce819c010 con 0x7f6ce8106710 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6ceeaf6640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 0x7f6ce819b8e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009fd0 tx=0x7f6ce000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cdf7fe640 1 -- 172.21.15.49:0/1641444346 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000ce20 con 0x7f6ce8106710 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cdf7fe640 1 -- 172.21.15.49:0/1641444346 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce0005020 con 0x7f6ce8106710 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce819c300 con 0x7f6ce8106710 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cdf7fe640 1 -- 172.21.15.49:0/1641444346 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0010720 con 0x7f6ce8106710 2024-03-16T08:25:11.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.862+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce81a3b90 con 0x7f6ce8106710 2024-03-16T08:25:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.865+0000 7f6cdf7fe640 1 -- 172.21.15.49:0/1641444346 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6ce0005540 con 0x7f6ce8106710 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.865+0000 7f6cf3212740 1 Processor -- start 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.865+0000 7f6cf3212740 1 -- start start 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca289e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 0x55fca289eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca28a7da0 0x55fca28aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cdf7fe640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 0x7f6cd40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fca28020d0 con 0x55fca28a7da0 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fca27aac70 con 0x55fca289ea30 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cf3212740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fca27fe170 con 0x55fca289f710 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca289e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6ceeaf6640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 0x7f6cd40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 0x55fca289eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca289e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:36972/0 (socket says 172.21.15.49:36972) 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cedaf4640 1 -- 172.21.15.49:0/1085124695 learned_addr learned my addr 172.21.15.49:0/1085124695 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca28a7da0 0x55fca28aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cdf7fe640 1 -- 172.21.15.49:0/1641444346 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f6ce00cf560 con 0x7f6ce8106710 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 -- 172.21.15.49:0/1085124695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 msgr2=0x55fca289e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca289e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 -- 172.21.15.49:0/1085124695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca28a7da0 msgr2=0x55fca28aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca28a7da0 0x55fca28aa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 -- 172.21.15.49:0/1085124695 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca25de7e0 con 0x55fca289ea30 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6ceeaf6640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 0x7f6cd40a4e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00cd2a0 tx=0x7f6ce00054b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.866+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 0x55fca289eeb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009c00 tx=0x7f6ce4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1013ae36b90eea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cdcbfe640 1 -- 172.21.15.49:0/1085124695 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4004050 con 0x55fca289ea30 2024-03-16T08:25:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cdcbfe640 1 -- 172.21.15.49:0/1085124695 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4005500 con 0x55fca289ea30 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cdcbfe640 1 -- 172.21.15.49:0/1085124695 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4004d10 con 0x55fca289ea30 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 -- 172.21.15.49:0/1085124695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 msgr2=0x55fca289eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 0x55fca289eeb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009c00 tx=0x7f6ce4011810 comp rx=0 tx=0).stop 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 -- 172.21.15.49:0/1085124695 shutdown_connections 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca28a7da0 0x55fca28aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289ea30 0x55fca289eeb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 --2- 172.21.15.49:0/1085124695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca289e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.867+0000 7f6cf3212740 1 -- 172.21.15.49:0/1085124695 >> 172.21.15.49:0/1085124695 conn(0x55fca2865c50 msgr2=0x55fca2899480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- 172.21.15.49:0/1085124695 shutdown_connections 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- 172.21.15.49:0/1085124695 wait complete. 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- start start 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 Processor -- start 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 0x55fca28a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fca28aac10 con 0x55fca289f710 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fca27be0d0 con 0x55fca28a7da0 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fca28aad90 con 0x55fca289ea30 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cee2f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.868+0000 7f6cf3212740 10 client.0 _ll_register_callbacks cb 0x55fca2787760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49076/0 (socket says 172.21.15.49:49076) 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 0x55fca28a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 -- 172.21.15.49:0/2594090631 learned_addr learned my addr 172.21.15.49:0/2594090631 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 -- 172.21.15.49:0/2594090631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 msgr2=0x55fca28a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 0x55fca28a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 -- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 msgr2=0x55fca28a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 -- 172.21.15.49:0/2594090631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca263ef70 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 0x55fca28a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011d10 tx=0x7f6ce4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b7a0885c3d0b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4029070 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f6ce4003a00 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce4005500 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6ce4035050 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cf3212740 10 client.17469 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.869+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fca24d6170 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4005500 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6ce4004fc0 con 0x55fca289f710 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 10 client.17469 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 20 client.17469 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca28b7c80 0x55fca28ba070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 4 client.17469 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca280f610 con 0x55fca28b7c80 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca28b7c80 0x55fca28ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cd1ffb640 10 client.17469 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:11.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.870+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca28b7c80 0x55fca28ba070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c843acf8852e609 server_cookie=55e4f88f7f62a6af in_seq=0 out_seq=0 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/2594090631 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fca280f610 con 0x55fca28b7c80 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cd1ffb640 10 client.17469 handle_command_reply: tid=0 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 client.17469 shutdown 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 msgr2=0x55fca28a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011d10 tx=0x7f6ce4005d70 comp rx=0 tx=0).stop 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 shutdown_connections 2024-03-16T08:25:11.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca28b7c80 0x55fca28ba070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289f710 0x55fca28a51d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 --2- 172.21.15.49:0/2594090631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289ea30 0x55fca28a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 >> 172.21.15.49:0/2594090631 conn(0x55fca2865c50 msgr2=0x55fca289b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cd1ffb640 0 client.17469 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 shutdown_connections 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 1 -- 172.21.15.49:0/2594090631 wait complete. 2024-03-16T08:25:11.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.871+0000 7f6cf3212740 20 client.17469 trim_cache size 0 max 16384 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 Processor -- start 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 -- start start 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 0x55fca28b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca28a7da0 0x55fca28bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81adbc0 con 0x55fca289ea30 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81ab490 con 0x55fca289f710 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cf3212740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81a9300 con 0x55fca28a7da0 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49082/0 (socket says 172.21.15.49:49082) 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.889+0000 7f6cedaf4640 1 -- 172.21.15.49:0/3711310256 learned_addr learned my addr 172.21.15.49:0/3711310256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 0x55fca28b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca28a7da0 0x55fca28bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 -- 172.21.15.49:0/3711310256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca28a7da0 msgr2=0x55fca28bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca28a7da0 0x55fca28bf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 -- 172.21.15.49:0/3711310256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 msgr2=0x55fca28b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 0x55fca28b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 -- 172.21.15.49:0/3711310256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca289da60 con 0x55fca289ea30 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 0x55fca28b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd802bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a6f4e79b96f2f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cd3fff640 1 -- 172.21.15.49:0/3711310256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80041a0 con 0x55fca289ea30 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cd3fff640 1 -- 172.21.15.49:0/3711310256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8004340 con 0x55fca289ea30 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.890+0000 7f6cd3fff640 1 -- 172.21.15.49:0/3711310256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8005620 con 0x55fca289ea30 2024-03-16T08:25:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- 172.21.15.49:0/3711310256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 msgr2=0x55fca28a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd802bc20 comp rx=0 tx=0).stop 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- 172.21.15.49:0/3711310256 shutdown_connections 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca28a7da0 0x55fca28bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca289f710 0x55fca28b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- 172.21.15.49:0/3711310256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28a73c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- 172.21.15.49:0/3711310256 >> 172.21.15.49:0/3711310256 conn(0x55fca2865c50 msgr2=0x55fca28a4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- 172.21.15.49:0/3711310256 shutdown_connections 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- 172.21.15.49:0/3711310256 wait complete. 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- start start 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 Processor -- start 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca28a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81a9950 con 0x55fca289ea30 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81aadc0 con 0x55fca28a7da0 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce81aa020 con 0x55fca289f710 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cf3212740 10 client.0 _ll_register_callbacks cb 0x7f6cd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cedaf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49098/0 (socket says 172.21.15.49:49098) 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cee2f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca28a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.891+0000 7f6cedaf4640 1 -- 172.21.15.49:0/894494404 learned_addr learned my addr 172.21.15.49:0/894494404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 -- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 msgr2=0x55fca28a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 -- 172.21.15.49:0/894494404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 msgr2=0x55fca28b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 -- 172.21.15.49:0/894494404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca291ced0 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cee2f5640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca28a16f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009250 tx=0x7f6ce4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ea2236793083e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce403e060 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f6ce4004650 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd8004340 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6ce4029050 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cf3212740 10 client.27358 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.892+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fca291e0c0 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8004340 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6ce4004d30 con 0x55fca289f710 2024-03-16T08:25:11.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 10 client.27358 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 20 client.27358 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca291e750 0x55fca2920b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 4 client.27358 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fca2809960 con 0x55fca291e750 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca291e750 0x55fca2920b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cd1ffb640 10 client.27358 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.893+0000 7f6cedaf4640 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca291e750 0x55fca2920b70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f45156e03f02f95 server_cookie=e701f114c4f0ec84 in_seq=0 out_seq=0 2024-03-16T08:25:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.894+0000 7f6cd1ffb640 1 -- 172.21.15.49:0/894494404 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fca2809960 con 0x55fca291e750 2024-03-16T08:25:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.894+0000 7f6cd1ffb640 10 client.27358 handle_command_reply: tid=0 2024-03-16T08:25:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.894+0000 7f6cf3212740 1 client.27358 shutdown 2024-03-16T08:25:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 msgr2=0x55fca28a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca28a16f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009250 tx=0x7f6ce4005f10 comp rx=0 tx=0).stop 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 shutdown_connections 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55fca291e750 0x55fca2920b70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fca28a7da0 0x55fca28a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fca289f710 0x55fca28a16f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 --2- 172.21.15.49:0/894494404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fca289ea30 0x55fca28b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 >> 172.21.15.49:0/894494404 conn(0x55fca2865c50 msgr2=0x55fca28984e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cd1ffb640 0 client.27358 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 shutdown_connections 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 1 -- 172.21.15.49:0/894494404 wait complete. 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf3212740 20 client.27358 trim_cache size 0 max 16384 2024-03-16T08:25:11.902 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:11.903 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:25:11.903 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "decb196a-5492-47ef-8b32-e503dc2fa880", 2024-03-16T08:25:11.903 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:25:11.903 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 msgr2=0x7f6cd40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 0x7f6cd40a4e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00cd2a0 tx=0x7f6ce00054b0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 msgr2=0x7f6ce819b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 0x7f6ce819b8e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009fd0 tx=0x7f6ce000ef10 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 shutdown_connections 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6cd40a2970 0x7f6cd40a4e30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce8106710 0x7f6ce819b8e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.895+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ce81010c0 0x7f6ce8197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.896+0000 7f6cf0559640 1 --2- 172.21.15.49:0/1641444346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce8100700 0x7f6ce8197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.896+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 >> 172.21.15.49:0/1641444346 conn(0x7f6ce8077840 msgr2=0x7f6ce8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.896+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 shutdown_connections 2024-03-16T08:25:11.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:11.896+0000 7f6cf0559640 1 -- 172.21.15.49:0/1641444346 wait complete. 2024-03-16T08:25:11.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:25:11.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 -- 172.21.15.49:0/2450902558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 msgr2=0x7f9d60107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 --2- 172.21.15.49:0/2450902558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60107b40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009930 tx=0x7f9d5002bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 -- 172.21.15.49:0/2450902558 shutdown_connections 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 --2- 172.21.15.49:0/2450902558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60102110 0x7f9d6010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 --2- 172.21.15.49:0/2450902558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60101750 0x7f9d60101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 --2- 172.21.15.49:0/2450902558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60107b40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 -- 172.21.15.49:0/2450902558 >> 172.21.15.49:0/2450902558 conn(0x7f9d600fd240 msgr2=0x7f9d600ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 -- 172.21.15.49:0/2450902558 shutdown_connections 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.016+0000 7f9d662d1640 1 -- 172.21.15.49:0/2450902558 wait complete. 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 Processor -- start 2024-03-16T08:25:12.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 -- start start 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 0x7f9d60073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60102110 0x7f9d60074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60111b70 con 0x7f9d60107760 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601119f0 con 0x7f9d60101750 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d662d1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60111cf0 con 0x7f9d60102110 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d6486e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 0x7f9d60073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d6486e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49120/0 (socket says 172.21.15.49:49120) 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d6486e640 1 -- 172.21.15.49:0/1539711119 learned_addr learned my addr 172.21.15.49:0/1539711119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.017+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60102110 0x7f9d60074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 -- 172.21.15.49:0/1539711119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60102110 msgr2=0x7f9d60074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60102110 0x7f9d60074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 -- 172.21.15.49:0/1539711119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 msgr2=0x7f9d60073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 0x7f9d60073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 -- 172.21.15.49:0/1539711119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601a65d0 con 0x7f9d60107760 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 0x7f9d60073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d6486e640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800eb10 tx=0x7f9d5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d5d7fa640 1 -- 172.21.15.49:0/1539711119 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004b20 con 0x7f9d60107760 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d5d7fa640 1 -- 172.21.15.49:0/1539711119 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58004cc0 con 0x7f9d60107760 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d5d7fa640 1 -- 172.21.15.49:0/1539711119 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58010760 con 0x7f9d60107760 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601a68c0 con 0x7f9d60107760 2024-03-16T08:25:12.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.018+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601a6dc0 con 0x7f9d60107760 2024-03-16T08:25:12.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5d7fa640 1 -- 172.21.15.49:0/1539711119 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9d58005020 con 0x7f9d60107760 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 Processor -- start 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 -- start start 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5d7fa640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 0x7f9d480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5604d4337500 con 0x5604d43d3a70 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604d42dfbe0 con 0x5604d43dc650 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d68f97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5604d43335a0 con 0x5604d43d3150 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 0x7f9d480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5d7fa640 1 -- 172.21.15.49:0/1539711119 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f9d580cf380 con 0x7f9d60107760 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49122/0 (socket says 172.21.15.49:49122) 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/597122330 learned_addr learned my addr 172.21.15.49:0/597122330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.021+0000 7f9d6486e640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/597122330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 msgr2=0x5604d43d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/597122330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 msgr2=0x5604d43dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/597122330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d42f34b0 con 0x5604d43d3a70 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d6486e640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 0x7f9d480a4e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005ab0 tx=0x7f9d50005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9d54011810 tx=0x7f9d540097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc8ec0bc2e1e2d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d467fc640 1 -- 172.21.15.49:0/597122330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004030 con 0x5604d43d3a70 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d467fc640 1 -- 172.21.15.49:0/597122330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5400fca0 con 0x5604d43d3a70 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.022+0000 7f9d467fc640 1 -- 172.21.15.49:0/597122330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54003a30 con 0x5604d43d3a70 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- 172.21.15.49:0/597122330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 msgr2=0x5604d43dc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9d54011810 tx=0x7f9d540097d0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- 172.21.15.49:0/597122330 shutdown_connections 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43dc110 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 --2- 172.21.15.49:0/597122330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- 172.21.15.49:0/597122330 >> 172.21.15.49:0/597122330 conn(0x5604d43cd880 msgr2=0x5604d43cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- 172.21.15.49:0/597122330 shutdown_connections 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- 172.21.15.49:0/597122330 wait complete. 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 -- start start 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.023+0000 7f9d68f97740 1 Processor -- start 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5604d43df440 con 0x5604d43d3150 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604d42f3160 con 0x5604d43dc650 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5604d43df5c0 con 0x5604d43d3a70 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 10 client.0 _ll_register_callbacks cb 0x5604d42d7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49126/0 (socket says 172.21.15.49:49126) 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 -- 172.21.15.49:0/2133408476 learned_addr learned my addr 172.21.15.49:0/2133408476 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d6486e640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 -- 172.21.15.49:0/2133408476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 msgr2=0x5604d43d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 -- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 msgr2=0x5604d43da150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43da150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 -- 172.21.15.49:0/2133408476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d41157d0 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9d50035000 tx=0x7f9d5002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985833242bacdddd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5004e070 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f9d50004f30 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.024+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5400fca0 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d50005130 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 10 client.17487 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5604d404b3f0 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5400fca0 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9d50034440 con 0x5604d43d3150 2024-03-16T08:25:12.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 10 client.17487 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 20 client.17487 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d43ec630 0x5604d43eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 4 client.17487 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d4344a40 con 0x5604d43ec630 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.025+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d43ec630 0x5604d43eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d43ec630 0x5604d43eea90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a826c87677c29 server_cookie=3b62c736b67f7b1d in_seq=0 out_seq=0 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d3f7fe640 10 client.17487 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/2133408476 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5604d4344a40 con 0x5604d43ec630 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d3f7fe640 10 client.17487 handle_command_reply: tid=0 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 client.17487 shutdown 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 msgr2=0x5604d43d96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9d50035000 tx=0x7f9d5002bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:12.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 shutdown_connections 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d43ec630 0x5604d43eea90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43da150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 --2- 172.21.15.49:0/2133408476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43d96d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d3f7fe640 0 client.17487 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.026+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 >> 172.21.15.49:0/2133408476 conn(0x5604d43cd880 msgr2=0x5604d43cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.027+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 shutdown_connections 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.027+0000 7f9d68f97740 1 -- 172.21.15.49:0/2133408476 wait complete. 2024-03-16T08:25:12.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.027+0000 7f9d68f97740 20 client.17487 trim_cache size 0 max 16384 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 Processor -- start 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 -- start start 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b2030 con 0x5604d43d3150 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601af900 con 0x5604d43dc650 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d68f97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601ad770 con 0x5604d43d3a70 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49142/0 (socket says 172.21.15.49:49142) 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.044+0000 7f9d5ffff640 1 -- 172.21.15.49:0/699095252 learned_addr learned my addr 172.21.15.49:0/699095252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d6486e640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 -- 172.21.15.49:0/699095252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 msgr2=0x5604d43d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 -- 172.21.15.49:0/699095252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 msgr2=0x5604d43e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 -- 172.21.15.49:0/699095252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d43d38a0 con 0x5604d43d3150 2024-03-16T08:25:12.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d6486e640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9d5002c170 tx=0x7f9d5002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8305b8ffd0c421 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d45ffb640 1 -- 172.21.15.49:0/699095252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5004e070 con 0x5604d43d3150 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d45ffb640 1 -- 172.21.15.49:0/699095252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50034640 con 0x5604d43d3150 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.045+0000 7f9d45ffb640 1 -- 172.21.15.49:0/699095252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003920 con 0x5604d43d3150 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- 172.21.15.49:0/699095252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 msgr2=0x5604d43cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9d5002c170 tx=0x7f9d5002cdd0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- 172.21.15.49:0/699095252 shutdown_connections 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3a70 0x5604d43d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 --2- 172.21.15.49:0/699095252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3150 0x5604d43cc070 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- 172.21.15.49:0/699095252 >> 172.21.15.49:0/699095252 conn(0x5604d43cd880 msgr2=0x5604d43d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- 172.21.15.49:0/699095252 shutdown_connections 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- 172.21.15.49:0/699095252 wait complete. 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 -- start start 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.046+0000 7f9d68f97740 1 Processor -- start 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601addc0 con 0x5604d43d3a70 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601af230 con 0x5604d43dc650 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601ae490 con 0x5604d43d3150 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 10 client.0 _ll_register_callbacks cb 0x7f9d50042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49154/0 (socket says 172.21.15.49:49154) 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/1762696259 learned_addr learned my addr 172.21.15.49:0/1762696259 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d6486e640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/1762696259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 msgr2=0x5604d43d61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 msgr2=0x5604d43d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43d6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 -- 172.21.15.49:0/1762696259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d400b0f0 con 0x5604d43d3a70 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d5f7fe640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d54003660 tx=0x7f9d54005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceae99a5171b3d30 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540099e0 con 0x5604d43d3a70 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f9d54009c60 con 0x5604d43d3a70 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d50034640 con 0x5604d43d3a70 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d54038050 con 0x5604d43d3a70 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 10 client.17493 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.047+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5604d42fe9d0 con 0x5604d43d3a70 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50034640 con 0x5604d43d3a70 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9d54033030 con 0x5604d43d3a70 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 10 client.17493 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 20 client.17493 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d4452de0 0x5604d4455200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 4 client.17493 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604d433ed90 con 0x5604d4452de0 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d4452de0 0x5604d4455200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d3f7fe640 10 client.17493 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.048+0000 7f9d5ffff640 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d4452de0 0x5604d4455200 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc4b3014e790543 server_cookie=8a7bcb47e0d0f468 in_seq=0 out_seq=0 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d3f7fe640 1 -- 172.21.15.49:0/1762696259 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604d433ed90 con 0x5604d4452de0 2024-03-16T08:25:12.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d3f7fe640 10 client.17493 handle_command_reply: tid=0 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 client.17493 shutdown 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 msgr2=0x5604d43d66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d54003660 tx=0x7f9d54005db0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 shutdown_connections 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5604d4452de0 0x5604d4455200 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604d43dc650 0x5604d43d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d3f7fe640 0 client.17493 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5604d43d3a70 0x5604d43d66e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 --2- 172.21.15.49:0/1762696259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604d43d3150 0x5604d43d61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 >> 172.21.15.49:0/1762696259 conn(0x5604d43cd880 msgr2=0x5604d43ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 shutdown_connections 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 1 -- 172.21.15.49:0/1762696259 wait complete. 2024-03-16T08:25:12.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d68f97740 20 client.17493 trim_cache size 0 max 16384 2024-03-16T08:25:12.059 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:25:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:25:12.059 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 msgr2=0x7f9d480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 0x7f9d480a4e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005ab0 tx=0x7f9d50005fd0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 msgr2=0x7f9d60074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.049+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800eb10 tx=0x7f9d5800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 shutdown_connections 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9d480a2970 0x7f9d480a4e30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d60107760 0x7f9d60074560 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d60102110 0x7f9d60074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 --2- 172.21.15.49:0/1539711119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d60101750 0x7f9d60073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 >> 172.21.15.49:0/1539711119 conn(0x7f9d600fd240 msgr2=0x7f9d600fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 shutdown_connections 2024-03-16T08:25:12.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.050+0000 7f9d662d1640 1 -- 172.21.15.49:0/1539711119 wait complete. 2024-03-16T08:25:12.062 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:25:12.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:25:12.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 -- 172.21.15.49:0/2232358677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 msgr2=0x7f297c10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 --2- 172.21.15.49:0/2232358677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c10d570 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2974009cd0 tx=0x7f297402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 -- 172.21.15.49:0/2232358677 shutdown_connections 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 --2- 172.21.15.49:0/2232358677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c10d570 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 --2- 172.21.15.49:0/2232358677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1006a0 0x7f297c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 --2- 172.21.15.49:0/2232358677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1066b0 0x7f297c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 -- 172.21.15.49:0/2232358677 >> 172.21.15.49:0/2232358677 conn(0x7f297c077780 msgr2=0x7f297c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 -- 172.21.15.49:0/2232358677 shutdown_connections 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.169+0000 7f2982c36640 1 -- 172.21.15.49:0/2232358677 wait complete. 2024-03-16T08:25:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 Processor -- start 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 -- start start 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 0x7f297c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1066b0 0x7f297c1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f297c110a80 con 0x7f297c101060 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f297c110900 con 0x7f297c1006a0 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f2982c36640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f297c110c00 con 0x7f297c1066b0 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49158/0 (socket says 172.21.15.49:49158) 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 -- 172.21.15.49:0/2266418342 learned_addr learned my addr 172.21.15.49:0/2266418342 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f29811d3640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1066b0 0x7f297c1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f29809d2640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 0x7f297c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 -- 172.21.15.49:0/2266418342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1066b0 msgr2=0x7f297c1a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1066b0 0x7f297c1a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 -- 172.21.15.49:0/2266418342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 msgr2=0x7f297c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 0x7f297c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f297bfff640 1 -- 172.21.15.49:0/2266418342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c1a43e0 con 0x7f297c101060 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.170+0000 7f29809d2640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 0x7f297c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f297bfff640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9f0 tx=0x7f296400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400cd80 con 0x7f297c101060 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1a45b0 con 0x7f297c101060 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2964010910 con 0x7f297c101060 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640188f0 con 0x7f297c101060 2024-03-16T08:25:12.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.171+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1a4ad0 con 0x7f297c101060 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.172+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c068a20 con 0x7f297c101060 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.173+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2964010ac0 con 0x7f297c101060 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.174+0000 7f2979ffb640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 0x7f29680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.174+0000 7f29809d2640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 0x7f29680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.174+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f29640cf2e0 con 0x7f297c101060 2024-03-16T08:25:12.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.174+0000 7f29809d2640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 0x7f29680a4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f296c009bb0 tx=0x7f296c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.178+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f296409c030 con 0x7f297c101060 2024-03-16T08:25:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: pgmap v245: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:25:12.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:25:12.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:12 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:25:12.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.400+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297c1a4d30 con 0x7f297c101060 2024-03-16T08:25:12.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.401+0000 7f2979ffb640 1 -- 172.21.15.49:0/2266418342 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f296409b930 con 0x7f297c101060 2024-03-16T08:25:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:25:12.403 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:25:12.403 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 msgr2=0x7f29680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 0x7f29680a4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f296c009bb0 tx=0x7f296c005f90 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 msgr2=0x7f297c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9f0 tx=0x7f296400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 shutdown_connections 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f29680a2970 0x7f29680a4e30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c1066b0 0x7f297c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297c101060 0x7f297c199360 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 --2- 172.21.15.49:0/2266418342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f297c1006a0 0x7f297c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 >> 172.21.15.49:0/2266418342 conn(0x7f297c077780 msgr2=0x7f297c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 shutdown_connections 2024-03-16T08:25:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.403+0000 7f2982c36640 1 -- 172.21.15.49:0/2266418342 wait complete. 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: pgmap v245: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.414 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:12 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:25:12.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: pgmap v245: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:12.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:12 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:25:12.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- 172.21.15.49:0/2333168493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 msgr2=0x7fdba410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/2333168493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba410d520 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009cd0 tx=0x7fdba002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- 172.21.15.49:0/2333168493 shutdown_connections 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/2333168493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba410d520 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/2333168493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4100650 0x7fdba4100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/2333168493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4106660 0x7fdba4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- 172.21.15.49:0/2333168493 >> 172.21.15.49:0/2333168493 conn(0x7fdba40777e0 msgr2=0x7fdba4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- 172.21.15.49:0/2333168493 shutdown_connections 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- 172.21.15.49:0/2333168493 wait complete. 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 Processor -- start 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.525+0000 7fdbab2f8640 1 -- start start 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4100650 0x7fdba4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4106660 0x7fdba419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4110a30 con 0x7fdba4101010 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41108b0 con 0x7fdba4100650 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4110bb0 con 0x7fdba4106660 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53478/0 (socket says 172.21.15.49:53478) 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba9094640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4100650 0x7fdba4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 -- 172.21.15.49:0/507065249 learned_addr learned my addr 172.21.15.49:0/507065249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba9895640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4106660 0x7fdba419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 -- 172.21.15.49:0/507065249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4106660 msgr2=0x7fdba419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4106660 0x7fdba419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 -- 172.21.15.49:0/507065249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4100650 msgr2=0x7fdba4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4100650 0x7fdba4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 -- 172.21.15.49:0/507065249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba419d010 con 0x7fdba4101010 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdba8893640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400e9f0 tx=0x7fdb9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/507065249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400cd80 con 0x7fdba4101010 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/507065249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb94010910 con 0x7fdba4101010 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba419d300 con 0x7fdba4101010 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.526+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/507065249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94018990 con 0x7fdba4101010 2024-03-16T08:25:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.527+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41a4b90 con 0x7fdba4101010 2024-03-16T08:25:12.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.528+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/507065249 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fdb94010ac0 con 0x7fdba4101010 2024-03-16T08:25:12.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 Processor -- start 2024-03-16T08:25:12.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 -- start start 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a475096940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 0x55a4750a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a474ffa490 con 0x55a475096e80 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a474fa2c90 con 0x55a4750a4140 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a474ff6530 con 0x55a475096560 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdb9a7fc640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 0x7fdb900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdba9094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdba9895640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 0x7fdb900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdba8893640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 0x55a4750a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdba9094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53486/0 (socket says 172.21.15.49:53486) 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.529+0000 7fdba9094640 1 -- 172.21.15.49:0/3161729636 learned_addr learned my addr 172.21.15.49:0/3161729636 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba9895640 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a475096940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/507065249 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdb940cf110 con 0x7fdba4101010 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 -- 172.21.15.49:0/3161729636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 msgr2=0x55a475096940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a475096940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 -- 172.21.15.49:0/3161729636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 msgr2=0x55a4750a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 -- 172.21.15.49:0/3161729636 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47505d500 con 0x55a4750a4140 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba9094640 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba9895640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 0x7fdb900a4e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005cc0 tx=0x7fdba002e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdba8893640 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 0x55a4750a6530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb940cc8f0 tx=0x7fdb9406b440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ee92df76209558 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdb8f7fe640 1 -- 172.21.15.49:0/3161729636 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9409b8b0 con 0x55a4750a4140 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdb8f7fe640 1 -- 172.21.15.49:0/3161729636 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9409f950 con 0x55a4750a4140 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.530+0000 7fdb8f7fe640 1 -- 172.21.15.49:0/3161729636 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940ccd00 con 0x55a4750a4140 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3161729636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 msgr2=0x55a4750a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 0x55a4750a6530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb940cc8f0 tx=0x7fdb9406b440 comp rx=0 tx=0).stop 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3161729636 shutdown_connections 2024-03-16T08:25:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a4750a4140 0x55a4750a6530 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096e80 0x55a4750a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3161729636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a475096940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3161729636 >> 172.21.15.49:0/3161729636 conn(0x55a475090f60 msgr2=0x55a475091340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3161729636 shutdown_connections 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3161729636 wait complete. 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 -- start start 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.531+0000 7fdbadfc3740 1 Processor -- start 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a4750a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4750a4140 0x55a4750a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4750a6e40 con 0x55a475096560 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a474fb6230 con 0x55a475096e80 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a4750a6fc0 con 0x55a4750a4140 2024-03-16T08:25:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdbadfc3740 10 client.0 _ll_register_callbacks cb 0x55a474ff2010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53502/0 (socket says 172.21.15.49:53502) 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 -- 172.21.15.49:0/2081524323 learned_addr learned my addr 172.21.15.49:0/2081524323 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba8893640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a4750a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9895640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4750a4140 0x55a4750a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 -- 172.21.15.49:0/2081524323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4750a4140 msgr2=0x55a4750a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4750a4140 0x55a4750a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 -- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 msgr2=0x55a4750a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a4750a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 -- 172.21.15.49:0/2081524323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a474fc5100 con 0x55a475096560 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdba9094640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c012700 tx=0x7fdb9c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b1ee866681341e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.532+0000 7fdb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c004050 con 0x55a475096560 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdb9c0041f0 con 0x55a475096560 2024-03-16T08:25:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9409f950 con 0x55a475096560 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdb9c0043f0 con 0x55a475096560 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 10 client.17511 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a474d0e3f0 con 0x55a475096560 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9409f950 con 0x55a475096560 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdb9c003b90 con 0x55a475096560 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 10 client.17511 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 20 client.17511 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a4750b4500 0x55a4750b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 4 client.17511 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4750079d0 con 0x55a4750b4500 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.533+0000 7fdba8893640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a4750b4500 0x55a4750b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.534+0000 7fdb8cff9640 10 client.17511 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.534+0000 7fdba8893640 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a4750b4500 0x55a4750b68f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55ffe8314fcbd19f server_cookie=1dcb6a9fb45d1277 in_seq=0 out_seq=0 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.534+0000 7fdb8cff9640 1 -- 172.21.15.49:0/2081524323 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a4750079d0 con 0x55a4750b4500 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.534+0000 7fdb8cff9640 10 client.17511 handle_command_reply: tid=0 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.534+0000 7fdbadfc3740 1 client.17511 shutdown 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 msgr2=0x55a4750a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c012700 tx=0x7fdb9c012bc0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 shutdown_connections 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a4750b4500 0x55a4750b68f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a4750a4140 0x55a4750a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a4750a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2081524323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a475096560 0x55a4750a1200 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdb8cff9640 0 client.17511 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 >> 172.21.15.49:0/2081524323 conn(0x55a475090f60 msgr2=0x55a475092cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 shutdown_connections 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2081524323 wait complete. 2024-03-16T08:25:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.535+0000 7fdbadfc3740 20 client.17511 trim_cache size 0 max 16384 2024-03-16T08:25:12.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 Processor -- start 2024-03-16T08:25:12.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 -- start start 2024-03-16T08:25:12.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096560 0x55a4750a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096e80 0x55a4750afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41adc10 con 0x55a4750a4140 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41ab4e0 con 0x55a475096560 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41a9330 con 0x55a475096e80 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdba9895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdba9094640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096560 0x55a4750a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdba9895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53510/0 (socket says 172.21.15.49:53510) 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.548+0000 7fdba9895640 1 -- 172.21.15.49:0/2584671560 learned_addr learned my addr 172.21.15.49:0/2584671560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba8893640 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096e80 0x55a4750afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 -- 172.21.15.49:0/2584671560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096e80 msgr2=0x55a4750afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096e80 0x55a4750afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 -- 172.21.15.49:0/2584671560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096560 msgr2=0x55a4750a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096560 0x55a4750a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 -- 172.21.15.49:0/2584671560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a474dd87d0 con 0x55a4750a4140 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdba9895640 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009a60 tx=0x7fdba0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2714bbc76816098e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdb8effd640 1 -- 172.21.15.49:0/2584671560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0039040 con 0x55a4750a4140 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdb8effd640 1 -- 172.21.15.49:0/2584671560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba0055070 con 0x55a4750a4140 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.549+0000 7fdb8effd640 1 -- 172.21.15.49:0/2584671560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0050460 con 0x55a4750a4140 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2584671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 msgr2=0x55a47508fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009a60 tx=0x7fdba0005c50 comp rx=0 tx=0).stop 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2584671560 shutdown_connections 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47508fe60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096e80 0x55a4750afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/2584671560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096560 0x55a4750a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2584671560 >> 172.21.15.49:0/2584671560 conn(0x55a475090f60 msgr2=0x55a47508f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2584671560 shutdown_connections 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- 172.21.15.49:0/2584671560 wait complete. 2024-03-16T08:25:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 -- start start 2024-03-16T08:25:12.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 Processor -- start 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.550+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a4750ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a47509dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41a99a0 con 0x55a4750a4140 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41aae10 con 0x55a475096e80 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41aa070 con 0x55a475096560 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 10 client.0 _ll_register_callbacks cb 0x7fdb9c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9094640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a4750ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53524/0 (socket says 172.21.15.49:53524) 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba8893640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a47509dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9094640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a4750ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:46732/0 (socket says 172.21.15.49:46732) 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 -- 172.21.15.49:0/3806315996 learned_addr learned my addr 172.21.15.49:0/3806315996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 -- 172.21.15.49:0/3806315996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 msgr2=0x55a4750ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a4750ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 -- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 msgr2=0x55a47509dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a47509dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 -- 172.21.15.49:0/3806315996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4751197f0 con 0x55a4750a4140 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdba9895640 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdba002cfa0 tx=0x7fdba0032b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5d12767d5d5ce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba005e060 con 0x55a4750a4140 2024-03-16T08:25:12.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdba0003b70 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.551+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba0055070 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdba0039040 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 10 client.17517 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a47511a970 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba0055070 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdba0034440 con 0x55a4750a4140 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 10 client.17517 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 20 client.17517 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a47511b030 0x55a47511d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 4 client.17517 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a475001d20 con 0x55a47511b030 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.552+0000 7fdba9094640 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a47511b030 0x55a47511d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdba9094640 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a47511b030 0x55a47511d420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c4ce2b4c63def11 server_cookie=6c50091043c785f8 in_seq=0 out_seq=0 2024-03-16T08:25:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdb8cff9640 10 client.17517 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdb8cff9640 1 -- 172.21.15.49:0/3806315996 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a475001d20 con 0x55a47511b030 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdb8cff9640 10 client.17517 handle_command_reply: tid=0 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 client.17517 shutdown 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 msgr2=0x55a47509e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdba002cfa0 tx=0x7fdba0032b70 comp rx=0 tx=0).stop 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 shutdown_connections 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55a47511b030 0x55a47511d420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4750a4140 0x55a47509e3a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a475096e80 0x55a47509dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdb8cff9640 0 client.17517 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 --2- 172.21.15.49:0/3806315996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a475096560 0x55a4750ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 >> 172.21.15.49:0/3806315996 conn(0x55a475090f60 msgr2=0x55a4750a1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 shutdown_connections 2024-03-16T08:25:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 1 -- 172.21.15.49:0/3806315996 wait complete. 2024-03-16T08:25:12.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.553+0000 7fdbadfc3740 20 client.17517 trim_cache size 0 max 16384 2024-03-16T08:25:12.561 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:12.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 msgr2=0x7fdb900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 0x7fdb900a4e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005cc0 tx=0x7fdba002e040 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 msgr2=0x7fdba4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400e9f0 tx=0x7fdb9400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 shutdown_connections 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdb900a2970 0x7fdb900a4e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdba4106660 0x7fdba419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4101010 0x7fdba4198560 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 --2- 172.21.15.49:0/507065249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4100650 0x7fdba4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 >> 172.21.15.49:0/507065249 conn(0x7fdba40777e0 msgr2=0x7fdba4068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 shutdown_connections 2024-03-16T08:25:12.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.554+0000 7fdbab2f8640 1 -- 172.21.15.49:0/507065249 wait complete. 2024-03-16T08:25:12.566 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:25:12.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- 172.21.15.49:0/3554023883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 msgr2=0x7fdf1c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3554023883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c100ad0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009930 tx=0x7fdf0c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:12.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- 172.21.15.49:0/3554023883 shutdown_connections 2024-03-16T08:25:12.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3554023883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 0x7fdf1c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3554023883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c100ad0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3554023883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 0x7fdf1c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- 172.21.15.49:0/3554023883 >> 172.21.15.49:0/3554023883 conn(0x7fdf1c0777e0 msgr2=0x7fdf1c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- 172.21.15.49:0/3554023883 shutdown_connections 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- 172.21.15.49:0/3554023883 wait complete. 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 Processor -- start 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.675+0000 7fdf243a9640 1 -- start start 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 0x7fdf1c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 0x7fdf1c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c111b80 con 0x7fdf1c100650 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c111a00 con 0x7fdf1c101010 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c111d00 con 0x7fdf1c106660 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53540/0 (socket says 172.21.15.49:53540) 2024-03-16T08:25:12.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 -- 172.21.15.49:0/3633214690 learned_addr learned my addr 172.21.15.49:0/3633214690 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf21944640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 0x7fdf1c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22946640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 0x7fdf1c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 -- 172.21.15.49:0/3633214690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 msgr2=0x7fdf1c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 0x7fdf1c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 -- 172.21.15.49:0/3633214690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 msgr2=0x7fdf1c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 0x7fdf1c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 -- 172.21.15.49:0/3633214690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c19d090 con 0x7fdf1c100650 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf22145640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9f0 tx=0x7fdf1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf137fe640 1 -- 172.21.15.49:0/3633214690 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400ccc0 con 0x7fdf1c100650 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf137fe640 1 -- 172.21.15.49:0/3633214690 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1400ce60 con 0x7fdf1c100650 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c19d380 con 0x7fdf1c100650 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf137fe640 1 -- 172.21.15.49:0/3633214690 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14010690 con 0x7fdf1c100650 2024-03-16T08:25:12.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.676+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a4c10 con 0x7fdf1c100650 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf137fe640 1 -- 172.21.15.49:0/3633214690 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fdf140046c0 con 0x7fdf1c100650 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 Processor -- start 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 -- start start 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb328490 con 0x55a6eb3c5150 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb2d0c90 con 0x55a6eb3cdd30 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb324530 con 0x55a6eb3c4830 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf21944640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf22946640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53548/0 (socket says 172.21.15.49:53548) 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.679+0000 7fdf22145640 1 -- 172.21.15.49:0/2214578594 learned_addr learned my addr 172.21.15.49:0/2214578594 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf137fe640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 0x7fdf080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 -- 172.21.15.49:0/2214578594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 msgr2=0x55a6eb3c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 -- 172.21.15.49:0/2214578594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 msgr2=0x55a6eb3cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 -- 172.21.15.49:0/2214578594 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb38b500 con 0x55a6eb3cdd30 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf22946640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 0x7fdf080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf137fe640 1 -- 172.21.15.49:0/3633214690 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdf140cf100 con 0x7fdf1c100650 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf21944640 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d0120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009230 tx=0x7fdf0c009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc07e8bee6a10f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf10bfe640 1 -- 172.21.15.49:0/2214578594 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c042070 con 0x55a6eb3cdd30 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf10bfe640 1 -- 172.21.15.49:0/2214578594 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c02c690 con 0x55a6eb3cdd30 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.680+0000 7fdf10bfe640 1 -- 172.21.15.49:0/2214578594 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c004b00 con 0x55a6eb3cdd30 2024-03-16T08:25:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- 172.21.15.49:0/2214578594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 msgr2=0x55a6eb3d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d0120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009230 tx=0x7fdf0c009610 comp rx=0 tx=0).stop 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf22946640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 0x7fdf080a4e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005fd0 tx=0x7fdf18004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- 172.21.15.49:0/2214578594 shutdown_connections 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d0120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/2214578594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- 172.21.15.49:0/2214578594 >> 172.21.15.49:0/2214578594 conn(0x55a6eb3bef60 msgr2=0x55a6eb3bf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- 172.21.15.49:0/2214578594 shutdown_connections 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- 172.21.15.49:0/2214578594 wait complete. 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 -- start start 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.681+0000 7fdf26f8c740 1 Processor -- start 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb3d0ac0 con 0x55a6eb3c4830 2024-03-16T08:25:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb2e4230 con 0x55a6eb3cdd30 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb3d0c40 con 0x55a6eb3c5150 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 10 client.0 _ll_register_callbacks cb 0x55a6eb320010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53554/0 (socket says 172.21.15.49:53554) 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 -- 172.21.15.49:0/3352437001 learned_addr learned my addr 172.21.15.49:0/3352437001 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf21944640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22946640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 -- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c5150 msgr2=0x55a6eb3cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 -- 172.21.15.49:0/3352437001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 msgr2=0x55a6eb3cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 -- 172.21.15.49:0/3352437001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6eb2f3100 con 0x55a6eb3c4830 2024-03-16T08:25:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22946640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf22145640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdf14097c50 tx=0x7fdf14009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e824b40fe2e71ec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1409b5c0 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdf140a3b00 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.682+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6eb3359d0 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf1409b150 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 10 client.17529 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6eb03c3f0 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a6eb3359d0 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdf140bb040 con 0x55a6eb3c4830 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 10 client.17529 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 10 client.17529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 20 client.17529 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb3de110 0x55a6eb3e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 4 client.17529 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6eb32fd20 con 0x55a6eb3de110 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.683+0000 7fdf21944640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb3de110 0x55a6eb3e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf21944640 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb3de110 0x55a6eb3e0500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=939396812eafe735 server_cookie=7a807ab127ee11b6 in_seq=0 out_seq=0 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf05ffb640 10 client.17529 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3352437001 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a6eb32fd20 con 0x55a6eb3de110 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf05ffb640 10 client.17529 handle_command_reply: tid=0 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 client.17529 shutdown 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 msgr2=0x55a6eb3cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdf14097c50 tx=0x7fdf14009720 comp rx=0 tx=0).stop 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 shutdown_connections 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb3de110 0x55a6eb3e0500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf05ffb640 0 client.17529 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c5150 0x55a6eb3cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3352437001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cad50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 >> 172.21.15.49:0/3352437001 conn(0x55a6eb3bef60 msgr2=0x55a6eb3c0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 shutdown_connections 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.684+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3352437001 wait complete. 2024-03-16T08:25:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.685+0000 7fdf26f8c740 20 client.17529 trim_cache size 0 max 16384 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 Processor -- start 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 -- start start 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1afcc0 con 0x55a6eb3c5150 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ad590 con 0x55a6eb3c4830 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ab400 con 0x55a6eb3cdd30 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53558/0 (socket says 172.21.15.49:53558) 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 -- 172.21.15.49:0/4225168468 learned_addr learned my addr 172.21.15.49:0/4225168468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf22145640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf22946640 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 -- 172.21.15.49:0/4225168468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 msgr2=0x55a6eb3e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 -- 172.21.15.49:0/4225168468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3c4830 msgr2=0x55a6eb3cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 -- 172.21.15.49:0/4225168468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb1067d0 con 0x55a6eb3c5150 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf22946640 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.698+0000 7fdf21944640 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005ec0 tx=0x7fdf0c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee59c996d823237 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf07fff640 1 -- 172.21.15.49:0/4225168468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c054070 con 0x55a6eb3c5150 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf07fff640 1 -- 172.21.15.49:0/4225168468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c04f3d0 con 0x55a6eb3c5150 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf07fff640 1 -- 172.21.15.49:0/4225168468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c042040 con 0x55a6eb3c5150 2024-03-16T08:25:12.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- 172.21.15.49:0/4225168468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 msgr2=0x55a6eb3d9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005ec0 tx=0x7fdf0c02e040 comp rx=0 tx=0).stop 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- 172.21.15.49:0/4225168468 shutdown_connections 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d9890 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/4225168468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3c4830 0x55a6eb3cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- 172.21.15.49:0/4225168468 >> 172.21.15.49:0/4225168468 conn(0x55a6eb3bef60 msgr2=0x55a6eb3bf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- 172.21.15.49:0/4225168468 shutdown_connections 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- 172.21.15.49:0/4225168468 wait complete. 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.699+0000 7fdf26f8c740 1 -- start start 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 Processor -- start 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1aba50 con 0x55a6eb3c5150 2024-03-16T08:25:12.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1acec0 con 0x55a6eb3cdd30 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ac120 con 0x55a6eb3c4830 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf26f8c740 10 client.0 _ll_register_callbacks cb 0x7fdf1400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf22946640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53570/0 (socket says 172.21.15.49:53570) 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 -- 172.21.15.49:0/3321281814 learned_addr learned my addr 172.21.15.49:0/3321281814 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf22145640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 -- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 msgr2=0x55a6eb3d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 -- 172.21.15.49:0/3321281814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 msgr2=0x55a6eb3d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 -- 172.21.15.49:0/3321281814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6eb4433b0 con 0x55a6eb3c5150 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf21944640 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c3b0 tx=0x7fdf0c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351d5e22e1a9d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.700+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c054070 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fdf0c004800 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c04f3d0 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c042040 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 10 client.17535 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6eb444690 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c04f3d0 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdf0c04f580 con 0x55a6eb3c5150 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 10 client.17535 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 20 client.17535 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb4449d0 0x55a6eb446dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 4 client.17535 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6eb2ad6a0 con 0x55a6eb4449d0 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.701+0000 7fdf22145640 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb4449d0 0x55a6eb446dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf05ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf22145640 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb4449d0 0x55a6eb446dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=889d76dd915a0c34 server_cookie=7c9517b3ce3bbe11 in_seq=0 out_seq=0 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf05ffb640 1 -- 172.21.15.49:0/3321281814 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6eb2ad6a0 con 0x55a6eb4449d0 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf05ffb640 10 client.17535 handle_command_reply: tid=0 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 client.17535 shutdown 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 msgr2=0x55a6eb3d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c3b0 tx=0x7fdf0c009680 comp rx=0 tx=0).stop 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 shutdown_connections 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a6eb4449d0 0x55a6eb446dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6eb3cdd30 0x55a6eb3d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf05ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6eb3c5150 0x55a6eb3d1d40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 --2- 172.21.15.49:0/3321281814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6eb3c4830 0x55a6eb3d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 >> 172.21.15.49:0/3321281814 conn(0x55a6eb3bef60 msgr2=0x55a6eb3cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 shutdown_connections 2024-03-16T08:25:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 1 -- 172.21.15.49:0/3321281814 wait complete. 2024-03-16T08:25:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf26f8c740 20 client.17535 trim_cache size 0 max 16384 2024-03-16T08:25:12.710 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 msgr2=0x7fdf080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.702+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 0x7fdf080a4e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005fd0 tx=0x7fdf18004a40 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 msgr2=0x7fdf1c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9f0 tx=0x7fdf1400edf0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 shutdown_connections 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdf080a2970 0x7fdf080a4e30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf1c106660 0x7fdf1c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf1c101010 0x7fdf1c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 --2- 172.21.15.49:0/3633214690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c100650 0x7fdf1c1980a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 >> 172.21.15.49:0/3633214690 conn(0x7fdf1c0777e0 msgr2=0x7fdf1c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 shutdown_connections 2024-03-16T08:25:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.703+0000 7fdf243a9640 1 -- 172.21.15.49:0/3633214690 wait complete. 2024-03-16T08:25:12.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 -- 172.21.15.49:0/2411267890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 msgr2=0x7fc11410e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/2411267890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11410e290 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009ca0 tx=0x7fc10c02c180 comp rx=0 tx=0).stop 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 -- 172.21.15.49:0/2411267890 shutdown_connections 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/2411267890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11410e290 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/2411267890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 0x7fc11410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/2411267890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 0x7fc114102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 -- 172.21.15.49:0/2411267890 >> 172.21.15.49:0/2411267890 conn(0x7fc114077840 msgr2=0x7fc114077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 -- 172.21.15.49:0/2411267890 shutdown_connections 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.823+0000 7fc11a0ea640 1 -- 172.21.15.49:0/2411267890 wait complete. 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 Processor -- start 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 -- start start 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 0x7fc114198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 0x7fc1141985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc114111bf0 con 0x7fc11410be20 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc114111a70 con 0x7fc114103370 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc11a0ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc114111d70 con 0x7fc114102a50 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.824+0000 7fc113fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53596/0 (socket says 172.21.15.49:53596) 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 -- 172.21.15.49:0/3458156044 learned_addr learned my addr 172.21.15.49:0/3458156044 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 0x7fc1141985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 0x7fc114198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 -- 172.21.15.49:0/3458156044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 msgr2=0x7fc114198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 0x7fc114198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 -- 172.21.15.49:0/3458156044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 msgr2=0x7fc1141985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 0x7fc1141985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 -- 172.21.15.49:0/3458156044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419d080 con 0x7fc11410be20 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc113fff640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc10c02c790 tx=0x7fc10c035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc110ff9640 1 -- 172.21.15.49:0/3458156044 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c039040 con 0x7fc11410be20 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419d310 con 0x7fc11410be20 2024-03-16T08:25:12.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc110ff9640 1 -- 172.21.15.49:0/3458156044 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c044070 con 0x7fc11410be20 2024-03-16T08:25:12.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc110ff9640 1 -- 172.21.15.49:0/3458156044 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c032940 con 0x7fc11410be20 2024-03-16T08:25:12.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.825+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a4c00 con 0x7fc11410be20 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc110ff9640 1 -- 172.21.15.49:0/3458156044 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc10c032ae0 con 0x7fc11410be20 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 Processor -- start 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 -- start start 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55574d929490 con 0x55574d9c6150 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55574d8d1c90 con 0x55574d9c5830 2024-03-16T08:25:12.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.828+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55574d925530 con 0x55574d9ced30 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53598/0 (socket says 172.21.15.49:53598) 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 -- 172.21.15.49:0/3521977873 learned_addr learned my addr 172.21.15.49:0/3521977873 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc110ff9640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 0x7fc0fc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 -- 172.21.15.49:0/3521977873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 msgr2=0x55574d9d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 -- 172.21.15.49:0/3521977873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 msgr2=0x55574d9c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 -- 172.21.15.49:0/3521977873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574d98c500 con 0x55574d9c6150 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc113fff640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 0x7fc0fc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc110ff9640 1 -- 172.21.15.49:0/3458156044 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fc10c0f10e0 con 0x7fc11410be20 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc104009d20 tx=0x7fc104011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe8a6df7ea7fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc0f9ffb640 1 -- 172.21.15.49:0/3521977873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10401e070 con 0x55574d9c6150 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc0f9ffb640 1 -- 172.21.15.49:0/3521977873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104019b10 con 0x55574d9c6150 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.829+0000 7fc0f9ffb640 1 -- 172.21.15.49:0/3521977873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400eb80 con 0x55574d9c6150 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3521977873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 msgr2=0x55574d9ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc113fff640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 0x7fc0fc0a4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0eea80 tx=0x7fc10c0a8040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc104009d20 tx=0x7fc104011810 comp rx=0 tx=0).stop 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3521977873 shutdown_connections 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c6150 0x55574d9ce7f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3521977873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3521977873 >> 172.21.15.49:0/3521977873 conn(0x55574d9bff60 msgr2=0x55574d9c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3521977873 shutdown_connections 2024-03-16T08:25:12.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3521977873 wait complete. 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.830+0000 7fc11cdb2740 1 -- start start 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 Processor -- start 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55574d9d1b20 con 0x55574d9c5830 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55574d8e5230 con 0x55574d9c6150 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55574d9d1ca0 con 0x55574d9ced30 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc11cdb2740 10 client.0 _ll_register_callbacks cb 0x55574d921010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53612/0 (socket says 172.21.15.49:53612) 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 -- 172.21.15.49:0/3107624102 learned_addr learned my addr 172.21.15.49:0/3107624102 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 -- 172.21.15.49:0/3107624102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 msgr2=0x55574d9cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 -- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 msgr2=0x55574d9cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc112ffd640 1 -- 172.21.15.49:0/3107624102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574d8f4100 con 0x55574d9c5830 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.831+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc112ffd640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc108012d10 tx=0x7fc108009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b831326c6b13e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10801a040 con 0x55574d9c5830 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fc108005220 con 0x55574d9c5830 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc104019b10 con 0x55574d9c5830 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc10800fe60 con 0x55574d9c5830 2024-03-16T08:25:12.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 10 client.17547 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55574d63d3f0 con 0x55574d9c5830 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104019b10 con 0x55574d9c5830 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc1080214e0 con 0x55574d9c5830 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 10 client.17547 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 20 client.17547 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574d9df180 0x55574d9e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 4 client.17547 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.832+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55574d9369d0 con 0x55574d9df180 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.833+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574d9df180 0x55574d9e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.833+0000 7fc0f2ffd640 10 client.17547 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.833+0000 7fc1137fe640 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574d9df180 0x55574d9e15e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe1c76d805486c2 server_cookie=9b221d71ccf043a0 in_seq=0 out_seq=0 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/3107624102 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55574d9369d0 con 0x55574d9df180 2024-03-16T08:25:12.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc0f2ffd640 10 client.17547 handle_command_reply: tid=0 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 client.17547 shutdown 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 msgr2=0x55574d9cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc108012d10 tx=0x7fc108009a40 comp rx=0 tx=0).stop 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 shutdown_connections 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574d9df180 0x55574d9e15e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc0f2ffd640 0 client.17547 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/3107624102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9cbd80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 >> 172.21.15.49:0/3107624102 conn(0x55574d9bff60 msgr2=0x55574d9c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 shutdown_connections 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 1 -- 172.21.15.49:0/3107624102 wait complete. 2024-03-16T08:25:12.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.834+0000 7fc11cdb2740 20 client.17547 trim_cache size 0 max 16384 2024-03-16T08:25:12.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.847+0000 7fc11cdb2740 1 Processor -- start 2024-03-16T08:25:12.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.847+0000 7fc11cdb2740 1 -- start start 2024-03-16T08:25:12.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9da920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141afcc0 con 0x55574d9c5830 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141ad590 con 0x55574d9c6150 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141ab400 con 0x55574d9ced30 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53624/0 (socket says 172.21.15.49:53624) 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 -- 172.21.15.49:0/300028099 learned_addr learned my addr 172.21.15.49:0/300028099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc1137fe640 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 -- 172.21.15.49:0/300028099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 msgr2=0x55574d9e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 -- 172.21.15.49:0/300028099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 msgr2=0x55574d9da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 -- 172.21.15.49:0/300028099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574d7077d0 con 0x55574d9c5830 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc112ffd640 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc108009f00 tx=0x7fc108009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7caee34a88de9b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc0f97fa640 1 -- 172.21.15.49:0/300028099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080122b0 con 0x55574d9c5830 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc0f97fa640 1 -- 172.21.15.49:0/300028099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc108021450 con 0x55574d9c5830 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.848+0000 7fc0f97fa640 1 -- 172.21.15.49:0/300028099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080035f0 con 0x55574d9c5830 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- 172.21.15.49:0/300028099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 msgr2=0x55574d9ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc108009f00 tx=0x7fc108009e90 comp rx=0 tx=0).stop 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- 172.21.15.49:0/300028099 shutdown_connections 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9ced30 0x55574d9e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c6150 0x55574d9da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/300028099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9c5830 0x55574d9ce4b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- 172.21.15.49:0/300028099 >> 172.21.15.49:0/300028099 conn(0x55574d9bff60 msgr2=0x55574d9cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- 172.21.15.49:0/300028099 shutdown_connections 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- 172.21.15.49:0/300028099 wait complete. 2024-03-16T08:25:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.849+0000 7fc11cdb2740 1 -- start start 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 Processor -- start 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9d6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9c6150 0x55574d9c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141aba50 con 0x55574d9ced30 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141acec0 con 0x55574d9c5830 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141ac120 con 0x55574d9c6150 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc11cdb2740 10 client.0 _ll_register_callbacks cb 0x7fc108012b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53640/0 (socket says 172.21.15.49:53640) 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 -- 172.21.15.49:0/2363949542 learned_addr learned my addr 172.21.15.49:0/2363949542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9d6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc1137fe640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9c6150 0x55574d9c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 -- 172.21.15.49:0/2363949542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9c6150 msgr2=0x55574d9c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9c6150 0x55574d9c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 -- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 msgr2=0x55574d9d6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9d6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 -- 172.21.15.49:0/2363949542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574d9d7690 con 0x55574d9ced30 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc112ffd640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc108001e90 tx=0x7fc108005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24a02e6e8e3cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.850+0000 7fc0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108021aa0 con 0x55574d9ced30 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7fc108021c40 con 0x55574d9ced30 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc108021450 con 0x55574d9ced30 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc1080035f0 con 0x55574d9ced30 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 10 client.17553 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55574da44fe0 con 0x55574d9ced30 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc108021450 con 0x55574d9ced30 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc10802aa00 con 0x55574d9ced30 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 10 client.17553 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 20 client.17553 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574da45590 0x55574da479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 4 client.17553 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55574d930d20 con 0x55574da45590 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.851+0000 7fc1137fe640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574da45590 0x55574da479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc0f2ffd640 10 client.17553 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc1137fe640 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574da45590 0x55574da479b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0273698fc88f2 server_cookie=7fec9cf18eb6ff6b in_seq=0 out_seq=0 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc0f2ffd640 1 -- 172.21.15.49:0/2363949542 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55574d930d20 con 0x55574da45590 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc0f2ffd640 10 client.17553 handle_command_reply: tid=0 2024-03-16T08:25:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 client.17553 shutdown 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 msgr2=0x55574d9c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc108001e90 tx=0x7fc108005d30 comp rx=0 tx=0).stop 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 shutdown_connections 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55574da45590 0x55574da479b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55574d9ced30 0x55574d9c85f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55574d9c6150 0x55574d9c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc0f2ffd640 0 client.17553 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 --2- 172.21.15.49:0/2363949542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55574d9c5830 0x55574d9d6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 >> 172.21.15.49:0/2363949542 conn(0x55574d9bff60 msgr2=0x55574d9befe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 shutdown_connections 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 1 -- 172.21.15.49:0/2363949542 wait complete. 2024-03-16T08:25:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.852+0000 7fc11cdb2740 20 client.17553 trim_cache size 0 max 16384 2024-03-16T08:25:12.861 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 msgr2=0x7fc0fc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 0x7fc0fc0a4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0eea80 tx=0x7fc10c0a8040 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 msgr2=0x7fc11419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc10c02c790 tx=0x7fc10c035e00 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 shutdown_connections 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc0fc0a2970 0x7fc0fc0a4e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11410be20 0x7fc11419c950 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc114103370 0x7fc1141985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 --2- 172.21.15.49:0/3458156044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc114102a50 0x7fc114198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 >> 172.21.15.49:0/3458156044 conn(0x7fc114077840 msgr2=0x7fc1140fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 shutdown_connections 2024-03-16T08:25:12.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:12.853+0000 7fc11a0ea640 1 -- 172.21.15.49:0/3458156044 wait complete. 2024-03-16T08:25:13.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:13 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2266418342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:13.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:13 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2266418342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:13.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:13 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2266418342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:13.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:25:13.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:25:13.866 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:25:13.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 -- 172.21.15.49:0/1168528611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8102140 msgr2=0x7f7db810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:13.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1168528611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8102140 0x7f7db810e650 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009cd0 tx=0x7f7db002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 -- 172.21.15.49:0/1168528611 shutdown_connections 2024-03-16T08:25:13.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1168528611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8102140 0x7f7db810e650 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1168528611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1168528611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8107790 0x7f7db8107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 -- 172.21.15.49:0/1168528611 >> 172.21.15.49:0/1168528611 conn(0x7f7db80fd210 msgr2=0x7f7db80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.990+0000 7f7dbf619640 1 -- 172.21.15.49:0/1168528611 shutdown_connections 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 -- 172.21.15.49:0/1168528611 wait complete. 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 Processor -- start 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 -- start start 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8102140 0x7f7db8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8111b60 con 0x7f7db8107790 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81119e0 con 0x7f7db8101780 2024-03-16T08:25:13.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbf619640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8111ce0 con 0x7f7db8102140 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.991+0000 7f7dbdbb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53660/0 (socket says 172.21.15.49:53660) 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbd3b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 -- 172.21.15.49:0/1687708399 learned_addr learned my addr 172.21.15.49:0/1687708399 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8102140 0x7f7db8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 -- 172.21.15.49:0/1687708399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8102140 msgr2=0x7f7db8074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8102140 0x7f7db8074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 -- 172.21.15.49:0/1687708399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 msgr2=0x7f7db8073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 -- 172.21.15.49:0/1687708399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db81a6520 con 0x7f7db8107790 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbdbb6640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7db00348f0 tx=0x7f7db0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dae7fc640 1 -- 172.21.15.49:0/1687708399 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0041020 con 0x7f7db8107790 2024-03-16T08:25:13.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dae7fc640 1 -- 172.21.15.49:0/1687708399 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db0032e80 con 0x7f7db8107790 2024-03-16T08:25:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db81a67b0 con 0x7f7db8107790 2024-03-16T08:25:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dae7fc640 1 -- 172.21.15.49:0/1687708399 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0039040 con 0x7f7db8107790 2024-03-16T08:25:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.992+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81a6ca0 con 0x7f7db8107790 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.995+0000 7f7dae7fc640 1 -- 172.21.15.49:0/1687708399 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7db0044020 con 0x7f7db8107790 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.995+0000 7f7dc2208740 1 Processor -- start 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 -- start start 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c866eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c86fda0 0x556b6c872190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556b6c7ca0d0 con 0x556b6c867710 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556b6c772c70 con 0x556b6c86fda0 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dc2208740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b6c7c6170 con 0x556b6c866a30 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dae7fc640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 0x7f7da40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53664/0 (socket says 172.21.15.49:53664) 2024-03-16T08:25:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/3982933590 learned_addr learned my addr 172.21.15.49:0/3982933590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbdbb6640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 0x7f7da40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c866eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dae7fc640 1 -- 172.21.15.49:0/1687708399 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f7db00f08b0 con 0x7f7db8107790 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c86fda0 0x556b6c872190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/3982933590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 msgr2=0x556b6c866eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c866eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/3982933590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c86fda0 msgr2=0x556b6c872190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c86fda0 0x556b6c872190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.996+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/3982933590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6c5a67e0 con 0x556b6c867710 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.997+0000 7f7dbdbb6640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 0x7f7da40a4e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db00ee1b0 tx=0x7f7db00b24a0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.997+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7da8013630 tx=0x7f7da8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401bd269c4b8175a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.997+0000 7f7da37fe640 1 -- 172.21.15.49:0/3982933590 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004250 con 0x556b6c867710 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.997+0000 7f7da37fe640 1 -- 172.21.15.49:0/3982933590 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da80043f0 con 0x556b6c867710 2024-03-16T08:25:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.997+0000 7f7da37fe640 1 -- 172.21.15.49:0/3982933590 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80119b0 con 0x556b6c867710 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- 172.21.15.49:0/3982933590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 msgr2=0x556b6c866460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7da8013630 tx=0x7f7da8009850 comp rx=0 tx=0).stop 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- 172.21.15.49:0/3982933590 shutdown_connections 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c86fda0 0x556b6c872190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c866eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- 172.21.15.49:0/3982933590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c866460 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- 172.21.15.49:0/3982933590 >> 172.21.15.49:0/3982933590 conn(0x556b6c82dc50 msgr2=0x556b6c861480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- 172.21.15.49:0/3982933590 shutdown_connections 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- 172.21.15.49:0/3982933590 wait complete. 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 -- start start 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 Processor -- start 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 0x556b6c86cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.998+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c86d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556b6c872c10 con 0x556b6c867710 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556b6c7860d0 con 0x556b6c866a30 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b6c872d90 con 0x556b6c86fda0 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 10 client.0 _ll_register_callbacks cb 0x556b6c74f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53674/0 (socket says 172.21.15.49:53674) 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 -- 172.21.15.49:0/4289205933 learned_addr learned my addr 172.21.15.49:0/4289205933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 0x556b6c86cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c86d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 -- 172.21.15.49:0/4289205933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 msgr2=0x556b6c86d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c86d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 -- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 msgr2=0x556b6c86cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 0x556b6c86cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 -- 172.21.15.49:0/4289205933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6c606f70 con 0x556b6c867710 2024-03-16T08:25:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 0x556b6c86cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7db4012d20 tx=0x7f7db400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3b50f4b7943d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7da0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4012110 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f7db4004050 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da80043f0 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7db40122b0 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 10 client.17565 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:13.999+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556b6c49e170 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da80043f0 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7db4010450 con 0x556b6c867710 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 10 client.17565 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 20 client.17565 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c87fef0 0x556b6c882320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 4 client.17565 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6c7d7610 con 0x556b6c87fef0 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.000+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c87fef0 0x556b6c882320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.001+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c87fef0 0x556b6c882320 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743ef345d0b0e3b8 server_cookie=6c75d7fca7985bfc in_seq=0 out_seq=0 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.001+0000 7f7da0ff9640 10 client.17565 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.001+0000 7f7da0ff9640 1 -- 172.21.15.49:0/4289205933 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556b6c7d7610 con 0x556b6c87fef0 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.001+0000 7f7da0ff9640 10 client.17565 handle_command_reply: tid=0 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.001+0000 7f7dc2208740 1 client.17565 shutdown 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 msgr2=0x556b6c86d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7db4012d20 tx=0x7f7db400f790 comp rx=0 tx=0).stop 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 shutdown_connections 2024-03-16T08:25:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c87fef0 0x556b6c882320 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c86d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c867710 0x556b6c86d270 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7da0ff9640 0 client.17565 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4289205933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c866a30 0x556b6c86cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 >> 172.21.15.49:0/4289205933 conn(0x556b6c82dc50 msgr2=0x556b6c862940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 shutdown_connections 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 1 -- 172.21.15.49:0/4289205933 wait complete. 2024-03-16T08:25:14.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.002+0000 7f7dc2208740 20 client.17565 trim_cache size 0 max 16384 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.019+0000 7f7dc2208740 1 Processor -- start 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 -- start start 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c86f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c87ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81b1f80 con 0x556b6c86fda0 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81af850 con 0x556b6c867710 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dc2208740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ad6c0 con 0x556b6c866a30 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53684/0 (socket says 172.21.15.49:53684) 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/4027880353 learned_addr learned my addr 172.21.15.49:0/4027880353 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c87ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c86f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/4027880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 msgr2=0x556b6c86f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c86f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/4027880353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 msgr2=0x556b6c87ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c87ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/4027880353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6c8829a0 con 0x556b6c86fda0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7da8006010 tx=0x7f7da8005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52eca15a0ef90dac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.020+0000 7f7da2ffd640 1 -- 172.21.15.49:0/4027880353 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80042b0 con 0x556b6c86fda0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7da2ffd640 1 -- 172.21.15.49:0/4027880353 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da8027040 con 0x556b6c86fda0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7da2ffd640 1 -- 172.21.15.49:0/4027880353 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8011710 con 0x556b6c86fda0 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- 172.21.15.49:0/4027880353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 msgr2=0x556b6c887c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7da8006010 tx=0x7f7da8005b90 comp rx=0 tx=0).stop 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- 172.21.15.49:0/4027880353 shutdown_connections 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c86fda0 0x556b6c887c30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c87ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 --2- 172.21.15.49:0/4027880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c866a30 0x556b6c86f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- 172.21.15.49:0/4027880353 >> 172.21.15.49:0/4027880353 conn(0x556b6c82dc50 msgr2=0x556b6c863160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- 172.21.15.49:0/4027880353 shutdown_connections 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- 172.21.15.49:0/4027880353 wait complete. 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 -- start start 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.021+0000 7f7dc2208740 1 Processor -- start 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c875750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c8e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81add10 con 0x556b6c866a30 2024-03-16T08:25:14.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81af180 con 0x556b6c867710 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ae3e0 con 0x556b6c86fda0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dc2208740 10 client.0 _ll_register_callbacks cb 0x7f7db4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53692/0 (socket says 172.21.15.49:53692) 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/932272815 learned_addr learned my addr 172.21.15.49:0/932272815 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c875750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c8e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/932272815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 msgr2=0x556b6c8e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c8e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 msgr2=0x556b6c875750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c875750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 -- 172.21.15.49:0/932272815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6c87c6d0 con 0x556b6c866a30 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7dbcbb4640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7da80115d0 tx=0x7f7da8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34969bfec55d4df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.022+0000 7f7da0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:14.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.024+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8003b50 con 0x556b6c866a30 2024-03-16T08:25:14.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.024+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da8027040 con 0x556b6c866a30 2024-03-16T08:25:14.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.024+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8536+0+0 (secure 0 0 0) 0x7f7da80040b0 con 0x556b6c866a30 2024-03-16T08:25:14.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.027+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7da80042b0 con 0x556b6c866a30 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.027+0000 7f7dc2208740 10 client.17571 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.027+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556b6c865a60 con 0x556b6c866a30 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da8027040 con 0x556b6c866a30 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7da8039070 con 0x556b6c866a30 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 10 client.17571 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 20 client.17571 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c8e7c20 0x556b6c8ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 4 client.17571 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b6c7d1960 con 0x556b6c8e7c20 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c8e7c20 0x556b6c8ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7dbd3b5640 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c8e7c20 0x556b6c8ea040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cae53b77f71d62 server_cookie=b640701227fca4fc in_seq=0 out_seq=0 2024-03-16T08:25:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.028+0000 7f7da0ff9640 10 client.17571 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7da0ff9640 1 -- 172.21.15.49:0/932272815 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b6c7d1960 con 0x556b6c8e7c20 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7da0ff9640 10 client.17571 handle_command_reply: tid=0 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 client.17571 shutdown 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 msgr2=0x556b6c875210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7da80115d0 tx=0x7f7da8001e50 comp rx=0 tx=0).stop 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 shutdown_connections 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556b6c8e7c20 0x556b6c8ea040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b6c86fda0 0x556b6c8e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b6c867710 0x556b6c875750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 --2- 172.21.15.49:0/932272815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b6c866a30 0x556b6c875210 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 >> 172.21.15.49:0/932272815 conn(0x556b6c82dc50 msgr2=0x556b6c86ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7da0ff9640 0 client.17571 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 shutdown_connections 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.030+0000 7f7dc2208740 1 -- 172.21.15.49:0/932272815 wait complete. 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.031+0000 7f7dc2208740 20 client.17571 trim_cache size 0 max 16384 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "c521264d-3b96-410e-91e8-9f81eebf2527", 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:25:14.033 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 msgr2=0x7f7da40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 0x7f7da40a4e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db00ee1b0 tx=0x7f7db00b24a0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 msgr2=0x7f7db8074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7db00348f0 tx=0x7f7db0032c90 comp rx=0 tx=0).stop 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 shutdown_connections 2024-03-16T08:25:14.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f7da40a29a0 0x7f7da40a4e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db8107790 0x7f7db8074560 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db8102140 0x7f7db8074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 --2- 172.21.15.49:0/1687708399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7db8101780 0x7f7db8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 >> 172.21.15.49:0/1687708399 conn(0x7f7db80fd210 msgr2=0x7f7db80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 shutdown_connections 2024-03-16T08:25:14.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.032+0000 7f7dbf619640 1 -- 172.21.15.49:0/1687708399 wait complete. 2024-03-16T08:25:14.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:25:14.045 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 -- 172.21.15.49:0/3369300318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4101050 msgr2=0x7f82c410d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 --2- 172.21.15.49:0/3369300318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4101050 0x7f82c410d560 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009cd0 tx=0x7f82bc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 -- 172.21.15.49:0/3369300318 shutdown_connections 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 --2- 172.21.15.49:0/3369300318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4101050 0x7f82c410d560 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 --2- 172.21.15.49:0/3369300318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100690 0x7f82c4100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 --2- 172.21.15.49:0/3369300318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 0x7f82c4106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 -- 172.21.15.49:0/3369300318 >> 172.21.15.49:0/3369300318 conn(0x7f82c4076a60 msgr2=0x7f82c4076e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 -- 172.21.15.49:0/3369300318 shutdown_connections 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 -- 172.21.15.49:0/3369300318 wait complete. 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.155+0000 7f82c973b640 1 Processor -- start 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 -- start start 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4101050 0x7f82c41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 0x7f82c419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4110ab0 con 0x7f82c4100690 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4110930 con 0x7f82c41066a0 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c973b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4110c30 con 0x7f82c4101050 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53716/0 (socket says 172.21.15.49:53716) 2024-03-16T08:25:14.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 -- 172.21.15.49:0/85032637 learned_addr learned my addr 172.21.15.49:0/85032637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c27fc640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4101050 0x7f82c41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c37fe640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 0x7f82c419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 -- 172.21.15.49:0/85032637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4101050 msgr2=0x7f82c41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4101050 0x7f82c41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 -- 172.21.15.49:0/85032637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 msgr2=0x7f82c419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 0x7f82c419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 -- 172.21.15.49:0/85032637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c419d080 con 0x7f82c4100690 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.156+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9f0 tx=0x7f82ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.157+0000 7f82bbfff640 1 -- 172.21.15.49:0/85032637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00cd80 con 0x7f82c4100690 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.157+0000 7f82bbfff640 1 -- 172.21.15.49:0/85032637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ac010910 con 0x7f82c4100690 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.157+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c419d370 con 0x7f82c4100690 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.157+0000 7f82bbfff640 1 -- 172.21.15.49:0/85032637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac018990 con 0x7f82c4100690 2024-03-16T08:25:14.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.157+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a4c00 con 0x7f82c4100690 2024-03-16T08:25:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.159+0000 7f82bbfff640 1 -- 172.21.15.49:0/85032637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f82ac010ac0 con 0x7f82c4100690 2024-03-16T08:25:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.159+0000 7f82cc3ff740 1 Processor -- start 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.159+0000 7f82cc3ff740 1 -- start start 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e61a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558a03dba500 con 0x558a03e5f650 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a03d62be0 con 0x558a03e56150 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a03db65a0 con 0x558a03e56a70 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e61a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:46906/0 (socket says 172.21.15.49:46906) 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 -- 172.21.15.49:0/3597842263 learned_addr learned my addr 172.21.15.49:0/3597842263 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82bbfff640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 0x7f82b00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 -- 172.21.15.49:0/3597842263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 msgr2=0x558a03e56530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e56530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 -- 172.21.15.49:0/3597842263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 msgr2=0x558a03e61a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e61a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c2ffd640 1 -- 172.21.15.49:0/3597842263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a03d764b0 con 0x558a03e56a70 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c37fe640 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c27fc640 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e61a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82c37fe640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 0x7f82b00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.160+0000 7f82bbfff640 1 -- 172.21.15.49:0/85032637 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f82ac0cf100 con 0x7f82c4100690 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0cc7e0 tx=0x7f82ac024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41d1355d1501356 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82b97fa640 1 -- 172.21.15.49:0/3597842263 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac026040 con 0x558a03e56a70 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82b97fa640 1 -- 172.21.15.49:0/3597842263 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ac09b830 con 0x558a03e56a70 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82b97fa640 1 -- 172.21.15.49:0/3597842263 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac09b190 con 0x558a03e56a70 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3597842263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 msgr2=0x558a03e5f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82c37fe640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 0x7f82b00a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005cc0 tx=0x7f82bc02e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.161+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0cc7e0 tx=0x7f82ac024000 comp rx=0 tx=0).stop 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3597842263 shutdown_connections 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e61a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5f110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3597842263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3597842263 >> 172.21.15.49:0/3597842263 conn(0x558a03e50880 msgr2=0x558a03e50c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3597842263 shutdown_connections 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3597842263 wait complete. 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- start start 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 Processor -- start 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e5f650 0x558a03e5d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558a03e623e0 con 0x558a03e56150 2024-03-16T08:25:14.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a03d76160 con 0x558a03e5f650 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a03e62560 con 0x558a03e56a70 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82cc3ff740 10 client.0 _ll_register_callbacks cb 0x558a03d5a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53740/0 (socket says 172.21.15.49:53740) 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82c2ffd640 1 -- 172.21.15.49:0/2306840918 learned_addr learned my addr 172.21.15.49:0/2306840918 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.162+0000 7f82c37fe640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e5f650 0x558a03e5d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 -- 172.21.15.49:0/2306840918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 msgr2=0x558a03e5cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 -- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e5f650 msgr2=0x558a03e5d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e5f650 0x558a03e5d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 -- 172.21.15.49:0/2306840918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a03b987d0 con 0x558a03e56150 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0a39f0 tx=0x7f82ac0d2d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ca1e18cdd6ec10 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac09b190 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f82ac097600 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ac09b830 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82ac09b600 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82cc3ff740 10 client.17583 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558a03ace3f0 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.163+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ac09b830 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f82ac09ba90 con 0x558a03e56150 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 10 client.17583 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 20 client.17583 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03e71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 4 client.17583 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:14.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a03dc7a40 con 0x558a03e6f470 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03e71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03e71860 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb4f78e1d00ec8c server_cookie=a1b97b5b29bfec62 in_seq=0 out_seq=0 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.164+0000 7f82aa7fc640 10 client.17583 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2306840918 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558a03dc7a40 con 0x558a03e6f470 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82aa7fc640 10 client.17583 handle_command_reply: tid=0 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 client.17583 shutdown 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 msgr2=0x558a03e5c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0a39f0 tx=0x7f82ac0d2d80 comp rx=0 tx=0).stop 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 shutdown_connections 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03e71860 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e5f650 0x558a03e5d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82aa7fc640 0 client.17583 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e5cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2306840918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56150 0x558a03e5c620 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 >> 172.21.15.49:0/2306840918 conn(0x558a03e50880 msgr2=0x558a03e525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 shutdown_connections 2024-03-16T08:25:14.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2306840918 wait complete. 2024-03-16T08:25:14.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.165+0000 7f82cc3ff740 20 client.17583 trim_cache size 0 max 16384 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 Processor -- start 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 -- start start 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e4b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41afdc0 con 0x558a03e5f650 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ad690 con 0x558a03e56150 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ab500 con 0x558a03e56a70 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53744/0 (socket says 172.21.15.49:53744) 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 -- 172.21.15.49:0/3836302768 learned_addr learned my addr 172.21.15.49:0/3836302768 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e4b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c27fc640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 -- 172.21.15.49:0/3836302768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 msgr2=0x558a03e6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e6b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 -- 172.21.15.49:0/3836302768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 msgr2=0x558a03e4b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e4b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c37fe640 1 -- 172.21.15.49:0/3836302768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a03a8e0f0 con 0x558a03e5f650 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.183+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e4b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:14.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82c37fe640 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009cd0 tx=0x7f82bc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6846a2064cb50a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82b8ff9640 1 -- 172.21.15.49:0/3836302768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc039040 con 0x558a03e5f650 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82b8ff9640 1 -- 172.21.15.49:0/3836302768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc053070 con 0x558a03e5f650 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82b8ff9640 1 -- 172.21.15.49:0/3836302768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc04e760 con 0x558a03e5f650 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3836302768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 msgr2=0x558a03e5cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009cd0 tx=0x7f82bc041000 comp rx=0 tx=0).stop 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3836302768 shutdown_connections 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e5f650 0x558a03e5cec0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e56a70 0x558a03e6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/3836302768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e4b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.184+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3836302768 >> 172.21.15.49:0/3836302768 conn(0x558a03e50880 msgr2=0x558a03e4f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3836302768 shutdown_connections 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- 172.21.15.49:0/3836302768 wait complete. 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- start start 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 Processor -- start 2024-03-16T08:25:14.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e67ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e5f650 0x558a03e63370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41abb50 con 0x558a03e56a70 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41acfc0 con 0x558a03e56150 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ac220 con 0x558a03e5f650 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82cc3ff740 10 client.0 _ll_register_callbacks cb 0x7f82b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53756/0 (socket says 172.21.15.49:53756) 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 -- 172.21.15.49:0/2115430185 learned_addr learned my addr 172.21.15.49:0/2115430185 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e67ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c37fe640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e5f650 0x558a03e63370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 -- 172.21.15.49:0/2115430185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e5f650 msgr2=0x558a03e63370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e5f650 0x558a03e63370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.185+0000 7f82c27fc640 1 -- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 msgr2=0x558a03e67ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e67ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82c27fc640 1 -- 172.21.15.49:0/2115430185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a03d819d0 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e67ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82c27fc640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x558a03e51a10 tx=0x7f82b4020610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce1a35b8fd41b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4010cf0 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f82b4004030 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc053070 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82b4004230 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 10 client.17589 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558a03ed5380 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc053070 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f82b4028410 con 0x558a03e56a70 2024-03-16T08:25:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 10 client.17589 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 20 client.17589 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03ed7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 4 client.17589 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.186+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a03dc1d90 con 0x558a03e6f470 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03ed7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82aa7fc640 10 client.17589 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82c2ffd640 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03ed7a90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecdc72f146e127 server_cookie=c039959a8836a3d3 in_seq=0 out_seq=0 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82aa7fc640 1 -- 172.21.15.49:0/2115430185 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558a03dc1d90 con 0x558a03e6f470 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82aa7fc640 10 client.17589 handle_command_reply: tid=0 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 client.17589 shutdown 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 msgr2=0x558a03e62e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x558a03e51a10 tx=0x7f82b4020610 comp rx=0 tx=0).stop 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 shutdown_connections 2024-03-16T08:25:14.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x558a03e6f470 0x558a03ed7a90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03e5f650 0x558a03e63370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82aa7fc640 0 client.17589 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a03e56a70 0x558a03e62e30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 --2- 172.21.15.49:0/2115430185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a03e56150 0x558a03e67ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 >> 172.21.15.49:0/2115430185 conn(0x558a03e50880 msgr2=0x558a03e52440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 shutdown_connections 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 1 -- 172.21.15.49:0/2115430185 wait complete. 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.187+0000 7f82cc3ff740 20 client.17589 trim_cache size 0 max 16384 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:14.196 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: "c521264d-3b96-410e-91e8-9f81eebf2527": { 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "c521264d-3b96-410e-91e8-9f81eebf2527", 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:25:14.197 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 msgr2=0x7f82b00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 0x7f82b00a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005cc0 tx=0x7f82bc02e040 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 msgr2=0x7f82c4198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9f0 tx=0x7f82ac00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 shutdown_connections 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f82b00a2970 0x7f82b00a4e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82c41066a0 0x7f82c419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4101050 0x7f82c41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 --2- 172.21.15.49:0/85032637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c4100690 0x7f82c4198090 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:14.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 >> 172.21.15.49:0/85032637 conn(0x7f82c4076a60 msgr2=0x7f82c406a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:14.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 shutdown_connections 2024-03-16T08:25:14.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:14.188+0000 7f82c973b640 1 -- 172.21.15.49:0/85032637 wait complete. 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: pgmap v246: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:25:14.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:14 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:14.200 INFO:tasks.cephfs.filesystem:scrub status for tag:c521264d-3b96-410e-91e8-9f81eebf2527 - {'path': '/', 'tag': 'c521264d-3b96-410e-91e8-9f81eebf2527', 'options': 'recursive,force'} 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: pgmap v246: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:25:14.376 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:14 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: pgmap v246: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:25:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T08:25:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:25:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:25:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:14 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:16.412 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:16 smithi129 ceph-mon[36561]: pgmap v248: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:25:16.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:16 smithi154 ceph-mon[36471]: pgmap v248: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:25:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:16 smithi049 ceph-mon[31942]: pgmap v248: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T08:25:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:17 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:25:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:17 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:25:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:17 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:25:18.477 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:18 smithi154 ceph-mon[36471]: pgmap v249: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 209 B/s wr, 7 op/s 2024-03-16T08:25:18.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:18 smithi049 ceph-mon[31942]: pgmap v249: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 209 B/s wr, 7 op/s 2024-03-16T08:25:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:18 smithi129 ceph-mon[36561]: pgmap v249: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 209 B/s wr, 7 op/s 2024-03-16T08:25:20.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:20 smithi154 ceph-mon[36471]: pgmap v250: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:20 smithi049 ceph-mon[31942]: pgmap v250: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:20.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:20 smithi129 ceph-mon[36561]: pgmap v250: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:22 smithi154 ceph-mon[36471]: pgmap v251: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:22 smithi049 ceph-mon[31942]: pgmap v251: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:22.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:22 smithi129 ceph-mon[36561]: pgmap v251: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:24 smithi154 ceph-mon[36471]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:24 smithi049 ceph-mon[31942]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:24 smithi129 ceph-mon[36561]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 102 B/s wr, 8 op/s 2024-03-16T08:25:25.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:25 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:25:25.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:25 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:25:25.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:25 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:25:26.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:26 smithi154 ceph-mon[36471]: pgmap v253: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:25:26.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:26 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:25:26.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:26 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:25:26.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:26 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:25:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:26 smithi049 ceph-mon[31942]: pgmap v253: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:25:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:26 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:25:26.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:26 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:25:26.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:26 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:25:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:26 smithi129 ceph-mon[36561]: pgmap v253: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:25:26.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:26 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:25:26.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:26 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:25:26.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:26 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:25:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:28 smithi154 ceph-mon[36471]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:28.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:28 smithi049 ceph-mon[31942]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:28.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:28 smithi129 ceph-mon[36561]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:30.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:30 smithi154 ceph-mon[36471]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:30.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:30 smithi049 ceph-mon[31942]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:30.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:30 smithi129 ceph-mon[36561]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:32.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:32 smithi154 ceph-mon[36471]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:32.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:32 smithi049 ceph-mon[31942]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:32.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:32 smithi129 ceph-mon[36561]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:34.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:34 smithi154 ceph-mon[36471]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:34.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:34 smithi049 ceph-mon[31942]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:34.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:34 smithi129 ceph-mon[36561]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:36.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:36 smithi154 ceph-mon[36471]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:36.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:36 smithi049 ceph-mon[31942]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:36.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:36 smithi129 ceph-mon[36561]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:38.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:38 smithi154 ceph-mon[36471]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:38.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:38 smithi049 ceph-mon[31942]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:38.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:38 smithi129 ceph-mon[36561]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:40.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:40 smithi154 ceph-mon[36471]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:40.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:40 smithi049 ceph-mon[31942]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:40.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:40 smithi129 ceph-mon[36561]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:42.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:42 smithi154 ceph-mon[36471]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:42.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:42 smithi049 ceph-mon[31942]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:42.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:42 smithi129 ceph-mon[36561]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:44.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:25:44.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:25:44.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 -- 172.21.15.49:0/3646268223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 msgr2=0x7f4b2410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/3646268223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b2410d520 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009cd0 tx=0x7f4b1c02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 -- 172.21.15.49:0/3646268223 shutdown_connections 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/3646268223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b2410d520 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/3646268223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/3646268223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b24106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 -- 172.21.15.49:0/3646268223 >> 172.21.15.49:0/3646268223 conn(0x7f4b240777e0 msgr2=0x7f4b24077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 -- 172.21.15.49:0/3646268223 shutdown_connections 2024-03-16T08:25:44.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.324+0000 7f4b28a4b640 1 -- 172.21.15.49:0/3646268223 wait complete. 2024-03-16T08:25:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 Processor -- start 2024-03-16T08:25:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 -- start start 2024-03-16T08:25:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b2419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24110a70 con 0x7f4b24101010 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241108f0 con 0x7f4b24100650 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b28a4b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24110bf0 con 0x7f4b24106660 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48880/0 (socket says 172.21.15.49:48880) 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b21d9b640 1 -- 172.21.15.49:0/392152631 learned_addr learned my addr 172.21.15.49:0/392152631 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b2259c640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.325+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b2419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 -- 172.21.15.49:0/392152631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 msgr2=0x7f4b2419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b2419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 -- 172.21.15.49:0/392152631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 msgr2=0x7f4b24196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 -- 172.21.15.49:0/392152631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2419bf00 con 0x7f4b24101010 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b2259c640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b2419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00e9f0 tx=0x7f4b0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/392152631 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00cd80 con 0x7f4b24101010 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/392152631 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c010910 con 0x7f4b24101010 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2419c1f0 con 0x7f4b24101010 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/392152631 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c018990 con 0x7f4b24101010 2024-03-16T08:25:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.326+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241a3a80 con 0x7f4b24101010 2024-03-16T08:25:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.328+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/392152631 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4b0c010ac0 con 0x7f4b24101010 2024-03-16T08:25:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.329+0000 7f4b2b63c740 1 Processor -- start 2024-03-16T08:25:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.329+0000 7f4b2b63c740 1 -- start start 2024-03-16T08:25:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.329+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d003b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0044110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0046a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b1b7fe640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 0x7f4b100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569cff9f500 con 0x5569d003b150 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5569cff47be0 con 0x5569d0044650 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5569cff9b5a0 con 0x5569d003ba70 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d003b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48896/0 (socket says 172.21.15.49:48896) 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b22d9d640 1 -- 172.21.15.49:0/3955796489 learned_addr learned my addr 172.21.15.49:0/3955796489 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0044110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 0x7f4b100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/392152631 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f4b0c0cf180 con 0x7f4b24101010 2024-03-16T08:25:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0046a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 -- 172.21.15.49:0/3955796489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 msgr2=0x5569d0046a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0046a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 -- 172.21.15.49:0/3955796489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 msgr2=0x5569d003b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d003b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 -- 172.21.15.49:0/3955796489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cff5b4b0 con 0x5569d003ba70 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0046a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 0x7f4b100a4e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c004af0 tx=0x7f4b1c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0044110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009f50 tx=0x7f4b140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869b9b92099903d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.330+0000 7f4b18ff9640 1 -- 172.21.15.49:0/3955796489 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14004030 con 0x5569d003ba70 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b18ff9640 1 -- 172.21.15.49:0/3955796489 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b14005500 con 0x5569d003ba70 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b18ff9640 1 -- 172.21.15.49:0/3955796489 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14004db0 con 0x5569d003ba70 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3955796489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 msgr2=0x5569d0044110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0044110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009f50 tx=0x7f4b140118d0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3955796489 shutdown_connections 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0046a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0044110 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3955796489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d003b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3955796489 >> 172.21.15.49:0/3955796489 conn(0x5569d0035880 msgr2=0x5569d0035c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3955796489 shutdown_connections 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.331+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3955796489 wait complete. 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 -- start start 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 Processor -- start 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0041b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d00420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569d00495c0 con 0x5569d003b150 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5569cff5b160 con 0x5569d0044650 2024-03-16T08:25:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5569d0049740 con 0x5569d003ba70 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2b63c740 10 client.0 _ll_register_callbacks cb 0x5569cff3f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48910/0 (socket says 172.21.15.49:48910) 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 -- 172.21.15.49:0/1259294987 learned_addr learned my addr 172.21.15.49:0/1259294987 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d00420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 -- 172.21.15.49:0/1259294987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 msgr2=0x5569d0041b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0041b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 -- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 msgr2=0x5569d00420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d00420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b2259c640 1 -- 172.21.15.49:0/1259294987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cfd7d7d0 con 0x5569d003b150 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.332+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0041b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2259c640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4b1401b040 tx=0x7f4b14005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89091b2f458d40b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14029020 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b14005500 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f4b1400f750 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b140046c0 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2b63c740 10 client.17601 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cfcb33f0 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b14005500 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b14025770 con 0x5569d003b150 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2b63c740 10 client.17601 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.333+0000 7f4b2b63c740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b2b63c740 20 client.17601 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00568a0 0x5569d0058c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b2b63c740 4 client.17601 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569cffaca40 con 0x5569d00568a0 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00568a0 0x5569d0058c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b09ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.334+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00568a0 0x5569d0058c90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7e39aca0024af7 server_cookie=f79a844ba460f216 in_seq=0 out_seq=0 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b09ffb640 1 -- 172.21.15.49:0/1259294987 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5569cffaca40 con 0x5569d00568a0 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b09ffb640 10 client.17601 handle_command_reply: tid=0 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 client.17601 shutdown 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 msgr2=0x5569d0041620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4b1401b040 tx=0x7f4b14005130 comp rx=0 tx=0).stop 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 shutdown_connections 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00568a0 0x5569d0058c90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d00420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0041b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b09ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/1259294987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0041620 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 >> 172.21.15.49:0/1259294987 conn(0x5569d0035880 msgr2=0x5569d00375d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 shutdown_connections 2024-03-16T08:25:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 1 -- 172.21.15.49:0/1259294987 wait complete. 2024-03-16T08:25:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.335+0000 7f4b2b63c740 20 client.17601 trim_cache size 0 max 16384 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 Processor -- start 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 -- start start 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d003ba70 0x5569d0040b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 0x5569d00524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241adb80 con 0x5569d003b150 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241ab450 con 0x5569d003ba70 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241a92c0 con 0x5569d0044650 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48912/0 (socket says 172.21.15.49:48912) 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b2259c640 1 -- 172.21.15.49:0/3818690473 learned_addr learned my addr 172.21.15.49:0/3818690473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.353+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d003ba70 0x5569d0040b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 0x5569d00524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 -- 172.21.15.49:0/3818690473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 msgr2=0x5569d00524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 0x5569d00524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 -- 172.21.15.49:0/3818690473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d003ba70 msgr2=0x5569d0040b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d003ba70 0x5569d0040b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 -- 172.21.15.49:0/3818690473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569d00405d0 con 0x5569d003b150 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 0x5569d00524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b2259c640 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4b140048a0 tx=0x7f4b14009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1776950d768d581f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b0bfff640 1 -- 172.21.15.49:0/3818690473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14029040 con 0x5569d003b150 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b0bfff640 1 -- 172.21.15.49:0/3818690473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b14005680 con 0x5569d003b150 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.354+0000 7f4b0bfff640 1 -- 172.21.15.49:0/3818690473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b140258b0 con 0x5569d003b150 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3818690473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 msgr2=0x5569d0043d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4b140048a0 tx=0x7f4b14009fb0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3818690473 shutdown_connections 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d0044650 0x5569d00524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d003ba70 0x5569d0040b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/3818690473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0043d50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3818690473 >> 172.21.15.49:0/3818690473 conn(0x5569d0035880 msgr2=0x5569d0036c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3818690473 shutdown_connections 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- 172.21.15.49:0/3818690473 wait complete. 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- start start 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 Processor -- start 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0047b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0048090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241a9910 con 0x5569d003b150 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241aad80 con 0x5569d0044650 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.355+0000 7f4b2b63c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241a9fe0 con 0x5569d003ba70 2024-03-16T08:25:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2b63c740 10 client.0 _ll_register_callbacks cb 0x7f4b0c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48918/0 (socket says 172.21.15.49:48918) 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 -- 172.21.15.49:0/2809131914 learned_addr learned my addr 172.21.15.49:0/2809131914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b22d9d640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0048090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0047b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 -- 172.21.15.49:0/2809131914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 msgr2=0x5569d0047b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0047b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 -- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 msgr2=0x5569d0048090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0048090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 -- 172.21.15.49:0/2809131914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cfc730f0 con 0x5569d003b150 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2259c640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4b14000c00 tx=0x7f4b14004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c590a02a1dd8122 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14029070 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f4b1400f750 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b14005680 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4b1400f950 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2b63c740 10 client.17607 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.356+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cff669d0 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b14005680 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4b14041910 con 0x5569d003b150 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 10 client.17607 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 20 client.17607 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00bedb0 0x5569d00c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 4 client.17607 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569cffa6d90 con 0x5569d00bedb0 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00bedb0 0x5569d00c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b09ffb640 10 client.17607 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b21d9b640 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00bedb0 0x5569d00c11a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc456901f107999 server_cookie=d43a9b91d6901863 in_seq=0 out_seq=0 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b09ffb640 1 -- 172.21.15.49:0/2809131914 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5569cffa6d90 con 0x5569d00bedb0 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b09ffb640 10 client.17607 handle_command_reply: tid=0 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.357+0000 7f4b2b63c740 1 client.17607 shutdown 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 msgr2=0x5569d0047610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4b14000c00 tx=0x7f4b14004970 comp rx=0 tx=0).stop 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 shutdown_connections 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5569d00bedb0 0x5569d00c11a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5569d0044650 0x5569d0048090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569d003ba70 0x5569d0047b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b09ffb640 0 client.17607 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 --2- 172.21.15.49:0/2809131914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569d003b150 0x5569d0047610 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 >> 172.21.15.49:0/2809131914 conn(0x5569d0035880 msgr2=0x5569d0034900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 shutdown_connections 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 1 -- 172.21.15.49:0/2809131914 wait complete. 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b2b63c740 20 client.17607 trim_cache size 0 max 16384 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:25:44.366 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 msgr2=0x7f4b100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 0x7f4b100a4e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c004af0 tx=0x7f4b1c005c50 comp rx=0 tx=0).stop 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 msgr2=0x7f4b24197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00e9f0 tx=0x7f4b0c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 shutdown_connections 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4b100a2970 0x7f4b100a4e30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b24106660 0x7f4b2419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b24101010 0x7f4b24197450 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 --2- 172.21.15.49:0/392152631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b24100650 0x7f4b24196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 >> 172.21.15.49:0/392152631 conn(0x7f4b240777e0 msgr2=0x7f4b24102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 shutdown_connections 2024-03-16T08:25:44.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.358+0000 7f4b28a4b640 1 -- 172.21.15.49:0/392152631 wait complete. 2024-03-16T08:25:44.371 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:25:44.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:25:44.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:44 smithi154 ceph-mon[36471]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 -- 172.21.15.49:0/1868027288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07481077e0 msgr2=0x7f0748107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 --2- 172.21.15.49:0/1868027288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07481077e0 0x7f0748107bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402bcd0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 -- 172.21.15.49:0/1868027288 shutdown_connections 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 --2- 172.21.15.49:0/1868027288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074806a830 0x7f074810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 --2- 172.21.15.49:0/1868027288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f074806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 --2- 172.21.15.49:0/1868027288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07481077e0 0x7f0748107bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.478+0000 7f074faea640 1 -- 172.21.15.49:0/1868027288 >> 172.21.15.49:0/1868027288 conn(0x7f0748077780 msgr2=0x7f0748077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- 172.21.15.49:0/1868027288 shutdown_connections 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- 172.21.15.49:0/1868027288 wait complete. 2024-03-16T08:25:44.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 Processor -- start 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- start start 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f074806a830 0x7f0748198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 0x7f074819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0748111ba0 con 0x7f0748069e90 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0748111a20 con 0x7f07481077e0 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074faea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0748111d20 con 0x7f074806a830 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074d886640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074d886640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48934/0 (socket says 172.21.15.49:48934) 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074d886640 1 -- 172.21.15.49:0/3331902834 learned_addr learned my addr 172.21.15.49:0/3331902834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074e087640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 0x7f074819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.479+0000 7f074d085640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f074806a830 0x7f0748198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 -- 172.21.15.49:0/3331902834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f074806a830 msgr2=0x7f0748198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f074806a830 0x7f0748198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 -- 172.21.15.49:0/3331902834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 msgr2=0x7f074819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 0x7f074819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 -- 172.21.15.49:0/3331902834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074819d030 con 0x7f0748069e90 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074e087640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 0x7f074819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074d886640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0744005940 tx=0x7f074402cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744041070 con 0x7f0748069e90 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074819d2c0 con 0x7f0748069e90 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744004450 con 0x7f0748069e90 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744003d80 con 0x7f0748069e90 2024-03-16T08:25:44.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.480+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a4bb0 con 0x7f0748069e90 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.481+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0748102f00 con 0x7f0748069e90 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.482+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0744044070 con 0x7f0748069e90 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.483+0000 7f073effd640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 0x7f07340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.483+0000 7f074d085640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 0x7f07340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.483+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f07440f0810 con 0x7f0748069e90 2024-03-16T08:25:44.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.483+0000 7f074d085640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 0x7f07340a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0748077df0 tx=0x7f0738005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.487+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07440bd050 con 0x7f0748069e90 2024-03-16T08:25:44.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:44 smithi049 ceph-mon[31942]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:44.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:44 smithi129 ceph-mon[36561]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.711+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0748068a20 con 0x7f0748069e90 2024-03-16T08:25:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.711+0000 7f073effd640 1 -- 172.21.15.49:0/3331902834 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f07440c10f0 con 0x7f0748069e90 2024-03-16T08:25:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:25:44.713 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:25:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 msgr2=0x7f07340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 0x7f07340a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0748077df0 tx=0x7f0738005f90 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 msgr2=0x7f0748198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0744005940 tx=0x7f074402cbd0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 shutdown_connections 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f07340a2970 0x7f07340a4e30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07481077e0 0x7f074819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f074806a830 0x7f0748198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.713+0000 7f074faea640 1 --2- 172.21.15.49:0/3331902834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0748069e90 0x7f0748198040 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.714+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 >> 172.21.15.49:0/3331902834 conn(0x7f0748077780 msgr2=0x7f07480ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.714+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 shutdown_connections 2024-03-16T08:25:44.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.714+0000 7f074faea640 1 -- 172.21.15.49:0/3331902834 wait complete. 2024-03-16T08:25:44.726 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- 172.21.15.49:0/1444345162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8100650 msgr2=0x7f77a8100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 --2- 172.21.15.49:0/1444345162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8100650 0x7f77a8100ad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f779c009930 tx=0x7f779c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- 172.21.15.49:0/1444345162 shutdown_connections 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 --2- 172.21.15.49:0/1444345162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8101010 0x7f77a810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 --2- 172.21.15.49:0/1444345162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8100650 0x7f77a8100ad0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 --2- 172.21.15.49:0/1444345162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 0x7f77a8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- 172.21.15.49:0/1444345162 >> 172.21.15.49:0/1444345162 conn(0x7f77a80777e0 msgr2=0x7f77a8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- 172.21.15.49:0/1444345162 shutdown_connections 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- 172.21.15.49:0/1444345162 wait complete. 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 Processor -- start 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.835+0000 7f77ada85640 1 -- start start 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8100650 0x7f77a8102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 0x7f77a8103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8110a70 con 0x7f77a8101010 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81108f0 con 0x7f77a8106660 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8110bf0 con 0x7f77a8100650 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48940/0 (socket says 172.21.15.49:48940) 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 -- 172.21.15.49:0/439397082 learned_addr learned my addr 172.21.15.49:0/439397082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a77fe640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8100650 0x7f77a8102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a7fff640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 0x7f77a8103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 -- 172.21.15.49:0/439397082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8100650 msgr2=0x7f77a8102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8100650 0x7f77a8102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 -- 172.21.15.49:0/439397082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 msgr2=0x7f77a8103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 0x7f77a8103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 -- 172.21.15.49:0/439397082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a81a6590 con 0x7f77a8101010 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f779c009230 tx=0x7f779c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a4ff9640 1 -- 172.21.15.49:0/439397082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c041070 con 0x7f77a8101010 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a81a6820 con 0x7f77a8101010 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a4ff9640 1 -- 172.21.15.49:0/439397082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c004450 con 0x7f77a8101010 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77a4ff9640 1 -- 172.21.15.49:0/439397082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c003e50 con 0x7f77a8101010 2024-03-16T08:25:44.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.836+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81a6d60 con 0x7f77a8101010 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a4ff9640 1 -- 172.21.15.49:0/439397082 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f779c044070 con 0x7f77a8101010 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 Processor -- start 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 -- start start 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e8f314d490 con 0x55e8f31f2d30 2024-03-16T08:25:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8f30f5c90 con 0x55e8f31ea150 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77b0675740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e8f3149530 con 0x55e8f31e9830 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48950/0 (socket says 172.21.15.49:48950) 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a6ffd640 1 -- 172.21.15.49:0/4082429678 learned_addr learned my addr 172.21.15.49:0/4082429678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.839+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:53542/0 (socket says 172.21.15.49:53542) 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a4ff9640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 0x7f77900a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a7fff640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 0x7f77900a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 -- 172.21.15.49:0/4082429678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 msgr2=0x55e8f31e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 -- 172.21.15.49:0/4082429678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 msgr2=0x55e8f31f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a4ff9640 1 -- 172.21.15.49:0/439397082 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f779c0f0600 con 0x7f77a8101010 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 -- 172.21.15.49:0/4082429678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f31b0500 con 0x55e8f31f2d30 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a77fe640 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a7fff640 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f779c0bcdd0 tx=0x7f779c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713a3a5506b017f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f778dffb640 1 -- 172.21.15.49:0/4082429678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c04b020 con 0x55e8f31f2d30 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f77a7fff640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 0x7f77900a4fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009fd0 tx=0x7f77a0004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f778dffb640 1 -- 172.21.15.49:0/4082429678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c004c80 con 0x55e8f31f2d30 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.840+0000 7f778dffb640 1 -- 172.21.15.49:0/4082429678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0c0ee0 con 0x55e8f31f2d30 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- 172.21.15.49:0/4082429678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 msgr2=0x55e8f31f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f779c0bcdd0 tx=0x7f779c000bc0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- 172.21.15.49:0/4082429678 shutdown_connections 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31f2d30 0x55e8f31f5120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31ea150 0x55e8f31f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 --2- 172.21.15.49:0/4082429678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- 172.21.15.49:0/4082429678 >> 172.21.15.49:0/4082429678 conn(0x55e8f31e3f60 msgr2=0x55e8f31e4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- 172.21.15.49:0/4082429678 shutdown_connections 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- 172.21.15.49:0/4082429678 wait complete. 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.841+0000 7f77b0675740 1 -- start start 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 Processor -- start 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31f2d30 0x55e8f31f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e8f31f58e0 con 0x55e8f31ea150 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8f3109230 con 0x55e8f31f2d30 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e8f31f5a60 con 0x55e8f31e9830 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77b0675740 10 client.0 _ll_register_callbacks cb 0x55e8f3145010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48960/0 (socket says 172.21.15.49:48960) 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 -- 172.21.15.49:0/346463129 learned_addr learned my addr 172.21.15.49:0/346463129 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a7fff640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31f2d30 0x55e8f31f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 -- 172.21.15.49:0/346463129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 msgr2=0x55e8f31efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 -- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31f2d30 msgr2=0x55e8f31f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31f2d30 0x55e8f31f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 -- 172.21.15.49:0/346463129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f3118100 con 0x55e8f31ea150 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ee370 tx=0x7f779c0eddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1403e46146fe3ede server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.842+0000 7f7786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0d2060 con 0x55e8f31ea150 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f779c0b94b0 con 0x55e8f31ea150 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c004c80 con 0x55e8f31ea150 2024-03-16T08:25:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f779c04b070 con 0x55e8f31ea150 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 10 client.17625 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8f2e613f0 con 0x55e8f31ea150 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c004c80 con 0x55e8f31ea150 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f779c0c50a0 con 0x55e8f31ea150 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 10 client.17625 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 10 client.17625 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 20 client.17625 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3202c20 0x55e8f3205010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 4 client.17625 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8f315a9d0 con 0x55e8f3202c20 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.843+0000 7f77a77fe640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3202c20 0x55e8f3205010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.844+0000 7f7786ffd640 10 client.17625 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.844+0000 7f77a77fe640 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3202c20 0x55e8f3205010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3939b4c5122fcdf1 server_cookie=94011228549c5c91 in_seq=0 out_seq=0 2024-03-16T08:25:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.844+0000 7f7786ffd640 1 -- 172.21.15.49:0/346463129 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e8f315a9d0 con 0x55e8f3202c20 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.844+0000 7f7786ffd640 10 client.17625 handle_command_reply: tid=0 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.844+0000 7f77b0675740 1 client.17625 shutdown 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 msgr2=0x55e8f31f02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ee370 tx=0x7f779c0eddf0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 shutdown_connections 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3202c20 0x55e8f3205010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31f2d30 0x55e8f31f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f7786ffd640 0 client.17625 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31f02d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 --2- 172.21.15.49:0/346463129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31e9830 0x55e8f31efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 >> 172.21.15.49:0/346463129 conn(0x55e8f31e3f60 msgr2=0x55e8f31e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 shutdown_connections 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 1 -- 172.21.15.49:0/346463129 wait complete. 2024-03-16T08:25:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.845+0000 7f77b0675740 20 client.17625 trim_cache size 0 max 16384 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 Processor -- start 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 -- start start 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f320a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b1ec0 con 0x55e8f31ea150 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81af790 con 0x55e8f31e9830 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77b0675740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ad600 con 0x55e8f31f2d30 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48964/0 (socket says 172.21.15.49:48964) 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1418823425 learned_addr learned my addr 172.21.15.49:0/1418823425 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77a77fe640 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.858+0000 7f77a7fff640 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f320a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1418823425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 msgr2=0x55e8f320a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f320a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1418823425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 msgr2=0x55e8f31f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1418823425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f2f2b7d0 con 0x55e8f31ea150 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f779c0bcd80 tx=0x7f779c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442479325c90eaf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f778d7fa640 1 -- 172.21.15.49:0/1418823425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0bc980 con 0x55e8f31ea150 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f778d7fa640 1 -- 172.21.15.49:0/1418823425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c0d5e50 con 0x55e8f31ea150 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f778d7fa640 1 -- 172.21.15.49:0/1418823425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0bc7a0 con 0x55e8f31ea150 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77b0675740 1 -- 172.21.15.49:0/1418823425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 msgr2=0x55e8f31fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.859+0000 7f77b0675740 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f779c0bcd80 tx=0x7f779c000bc0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- 172.21.15.49:0/1418823425 shutdown_connections 2024-03-16T08:25:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f320a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31fe5b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- 172.21.15.49:0/1418823425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- 172.21.15.49:0/1418823425 >> 172.21.15.49:0/1418823425 conn(0x55e8f31e3f60 msgr2=0x55e8f31f05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- 172.21.15.49:0/1418823425 shutdown_connections 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- 172.21.15.49:0/1418823425 wait complete. 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- start start 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 Processor -- start 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31ed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f3261e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81adc50 con 0x55e8f31ea150 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81af0c0 con 0x55e8f31e9830 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ae320 con 0x55e8f31f2d30 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77b0675740 10 client.0 _ll_register_callbacks cb 0x7f77980097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48970/0 (socket says 172.21.15.49:48970) 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1350621925 learned_addr learned my addr 172.21.15.49:0/1350621925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.860+0000 7f77a7fff640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f3261e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1350621925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 msgr2=0x55e8f3261e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f3261e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 msgr2=0x55e8f31ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 -- 172.21.15.49:0/1350621925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f3262460 con 0x55e8f31ea150 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a77fe640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ee370 tx=0x7f779c0bc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8994e9f1128cd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f7786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c082060 con 0x55e8f31ea150 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f779c0f6320 con 0x55e8f31ea150 2024-03-16T08:25:44.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c0d5e50 con 0x55e8f31ea150 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f779c004c80 con 0x55e8f31ea150 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77b0675740 10 client.17631 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8f3262460 con 0x55e8f31ea150 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.861+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c0d5e50 con 0x55e8f31ea150 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f779c0c50a0 con 0x55e8f31ea150 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 10 client.17631 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 10 client.17631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 20 client.17631 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3269ac0 0x55e8f326bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 4 client.17631 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8f3154d20 con 0x55e8f3269ac0 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77a77fe640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3269ac0 0x55e8f326bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f7786ffd640 10 client.17631 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77a77fe640 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3269ac0 0x55e8f326bee0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31c02e32b9322a13 server_cookie=5a67264f6a6d24cf in_seq=0 out_seq=0 2024-03-16T08:25:44.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f7786ffd640 1 -- 172.21.15.49:0/1350621925 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8f3154d20 con 0x55e8f3269ac0 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f7786ffd640 10 client.17631 handle_command_reply: tid=0 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 1 client.17631 shutdown 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 msgr2=0x55e8f31ed6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.862+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ee370 tx=0x7f779c0bc7a0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 shutdown_connections 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x55e8f3269ac0 0x55e8f326bee0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8f31f2d30 0x55e8f3261e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e8f31ea150 0x55e8f31ed6c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f7786ffd640 0 client.17631 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 --2- 172.21.15.49:0/1350621925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8f31e9830 0x55e8f31ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 >> 172.21.15.49:0/1350621925 conn(0x55e8f31e3f60 msgr2=0x55e8f31e2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 shutdown_connections 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 1 -- 172.21.15.49:0/1350621925 wait complete. 2024-03-16T08:25:44.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77b0675740 20 client.17631 trim_cache size 0 max 16384 2024-03-16T08:25:44.871 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:44.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 msgr2=0x7f77900a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 0x7f77900a4fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009fd0 tx=0x7f77a0004ab0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 msgr2=0x7f77a8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f779c009230 tx=0x7f779c02cb30 comp rx=0 tx=0).stop 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 shutdown_connections 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f77900a2b10 0x7f77900a4fd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8106660 0x7f77a8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77a8101010 0x7f77a8103130 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 --2- 172.21.15.49:0/439397082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a8100650 0x7f77a8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 >> 172.21.15.49:0/439397082 conn(0x7f77a80777e0 msgr2=0x7f77a8068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 shutdown_connections 2024-03-16T08:25:44.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.863+0000 7f77ada85640 1 -- 172.21.15.49:0/439397082 wait complete. 2024-03-16T08:25:44.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 -- 172.21.15.49:0/3187461194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d40693a0 msgr2=0x7f76d410e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3187461194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d40693a0 0x7f76d410e640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009d00 tx=0x7f76c402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 -- 172.21.15.49:0/3187461194 shutdown_connections 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3187461194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d40693a0 0x7f76d410e640 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3187461194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40689e0 0x7f76d4068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3187461194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d4107830 0x7f76d4107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.984+0000 7f76d95ab640 1 -- 172.21.15.49:0/3187461194 >> 172.21.15.49:0/3187461194 conn(0x7f76d40777e0 msgr2=0x7f76d4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.985+0000 7f76d95ab640 1 -- 172.21.15.49:0/3187461194 shutdown_connections 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.985+0000 7f76d95ab640 1 -- 172.21.15.49:0/3187461194 wait complete. 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.985+0000 7f76d95ab640 1 Processor -- start 2024-03-16T08:25:44.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.985+0000 7f76d95ab640 1 -- start start 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.985+0000 7f76d95ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d40689e0 0x7f76d4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4107830 0x7f76d419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111b80 con 0x7f76d40693a0 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111a00 con 0x7f76d4107830 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111d00 con 0x7f76d40689e0 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49000/0 (socket says 172.21.15.49:49000) 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 -- 172.21.15.49:0/3074536735 learned_addr learned my addr 172.21.15.49:0/3074536735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d40689e0 0x7f76d4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d37fe640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4107830 0x7f76d419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 -- 172.21.15.49:0/3074536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d40689e0 msgr2=0x7f76d4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d40689e0 0x7f76d4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 -- 172.21.15.49:0/3074536735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4107830 msgr2=0x7f76d419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4107830 0x7f76d419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 -- 172.21.15.49:0/3074536735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d419d010 con 0x7f76d40693a0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d27fc640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f76c800ea60 tx=0x7f76c800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76bbfff640 1 -- 172.21.15.49:0/3074536735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c800cde0 con 0x7f76d40693a0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d419d300 con 0x7f76d40693a0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76bbfff640 1 -- 172.21.15.49:0/3074536735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c80035f0 con 0x7f76d40693a0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76bbfff640 1 -- 172.21.15.49:0/3074536735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8010670 con 0x7f76d40693a0 2024-03-16T08:25:44.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.986+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a4b90 con 0x7f76d40693a0 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.988+0000 7f76bbfff640 1 -- 172.21.15.49:0/3074536735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f76c80037a0 con 0x7f76d40693a0 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 Processor -- start 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 -- start start 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735128c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735134120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b73508c490 con 0x55b735129150 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b735034c90 con 0x55b735128830 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76dc18a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b735088530 con 0x55b735131d30 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76bbfff640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 0x7f76bc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49014/0 (socket says 172.21.15.49:49014) 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 -- 172.21.15.49:0/3505570082 learned_addr learned my addr 172.21.15.49:0/3505570082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d37fe640 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735128c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d27fc640 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76bbfff640 1 -- 172.21.15.49:0/3074536735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f76c80cf0d0 con 0x7f76d40693a0 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d37fe640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 0x7f76bc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 -- 172.21.15.49:0/3505570082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 msgr2=0x55b735134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 -- 172.21.15.49:0/3505570082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 msgr2=0x55b735128c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:44.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735128c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.989+0000 7f76d2ffd640 1 -- 172.21.15.49:0/3505570082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7350ef500 con 0x55b735129150 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.990+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009cf0 tx=0x7f76cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf47634c72763eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.990+0000 7f76b97fa640 1 -- 172.21.15.49:0/3505570082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc004230 con 0x55b735129150 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.990+0000 7f76d37fe640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 0x7f76bc0a4e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005e30 tx=0x7f76c402c0f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.990+0000 7f76b97fa640 1 -- 172.21.15.49:0/3505570082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc0043d0 con 0x55b735129150 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.990+0000 7f76b97fa640 1 -- 172.21.15.49:0/3505570082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0198c0 con 0x55b735129150 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- 172.21.15.49:0/3505570082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 msgr2=0x55b7351317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009cf0 tx=0x7f76cc011810 comp rx=0 tx=0).stop 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- 172.21.15.49:0/3505570082 shutdown_connections 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b7351317f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- 172.21.15.49:0/3505570082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735128c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- 172.21.15.49:0/3505570082 >> 172.21.15.49:0/3505570082 conn(0x55b735122f60 msgr2=0x55b735123340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- 172.21.15.49:0/3505570082 shutdown_connections 2024-03-16T08:25:44.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- 172.21.15.49:0/3505570082 wait complete. 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- start start 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 Processor -- start 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b73512ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735129150 0x55b73512f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b735134b20 con 0x55b735131d30 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b735048230 con 0x55b735128830 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b735134ca0 con 0x55b735129150 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76dc18a740 10 client.0 _ll_register_callbacks cb 0x55b735084010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49024/0 (socket says 172.21.15.49:49024) 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76d37fe640 1 -- 172.21.15.49:0/2142419454 learned_addr learned my addr 172.21.15.49:0/2142419454 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b73512ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.991+0000 7f76d27fc640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735129150 0x55b73512f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 -- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735129150 msgr2=0x55b73512f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735129150 0x55b73512f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 -- 172.21.15.49:0/2142419454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 msgr2=0x55b73512ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b73512ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 -- 172.21.15.49:0/2142419454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b735057100 con 0x55b735131d30 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b73512ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76d37fe640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76c4036000 tx=0x7f76c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb64cf72a154733 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4053060 con 0x55b735131d30 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f76c4004040 con 0x55b735131d30 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76cc0043d0 con 0x55b735131d30 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f76c4047070 con 0x55b735131d30 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76dc18a740 10 client.17643 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:45.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b734da03f0 con 0x55b735131d30 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.992+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc0043d0 con 0x55b735131d30 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f76c403f070 con 0x55b735131d30 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 10 client.17643 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 10 client.17643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 20 client.17643 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351420b0 0x55b735144500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 4 client.17643 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7350999d0 con 0x55b7351420b0 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351420b0 0x55b735144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76b6ff5640 10 client.17643 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.993+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351420b0 0x55b735144500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5a4b6032ac2dcf7 server_cookie=87e7cedae7de3341 in_seq=0 out_seq=0 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76b6ff5640 1 -- 172.21.15.49:0/2142419454 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b7350999d0 con 0x55b7351420b0 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76b6ff5640 10 client.17643 handle_command_reply: tid=0 2024-03-16T08:25:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 client.17643 shutdown 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 msgr2=0x55b73512f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76c4036000 tx=0x7f76c40095c0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 shutdown_connections 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351420b0 0x55b735144500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735131d30 0x55b73512f7e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735129150 0x55b73512f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76b6ff5640 0 client.17643 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 --2- 172.21.15.49:0/2142419454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b73512ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 >> 172.21.15.49:0/2142419454 conn(0x55b735122f60 msgr2=0x55b735124cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 shutdown_connections 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 1 -- 172.21.15.49:0/2142419454 wait complete. 2024-03-16T08:25:45.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:44.994+0000 7f76dc18a740 20 client.17643 trim_cache size 0 max 16384 2024-03-16T08:25:45.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.007+0000 7f76dc18a740 1 Processor -- start 2024-03-16T08:25:45.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.007+0000 7f76dc18a740 1 -- start start 2024-03-16T08:25:45.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735131490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b73513d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41afdf0 con 0x55b735129150 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ad6c0 con 0x55b735128830 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76dc18a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ab530 con 0x55b735131d30 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49032/0 (socket says 172.21.15.49:49032) 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 -- 172.21.15.49:0/511321565 learned_addr learned my addr 172.21.15.49:0/511321565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d37fe640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b73513d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735131490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 -- 172.21.15.49:0/511321565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 msgr2=0x55b73513d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b73513d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 -- 172.21.15.49:0/511321565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 msgr2=0x55b735131490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735131490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 -- 172.21.15.49:0/511321565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b734e6a7d0 con 0x55b735129150 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735131490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76d27fc640 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c760 tx=0x7f76c806e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2fa4393cd04941 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76b8ff9640 1 -- 172.21.15.49:0/511321565 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c809be50 con 0x55b735129150 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76b8ff9640 1 -- 172.21.15.49:0/511321565 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c809b750 con 0x55b735129150 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.008+0000 7f76b8ff9640 1 -- 172.21.15.49:0/511321565 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8097dc0 con 0x55b735129150 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- 172.21.15.49:0/511321565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 msgr2=0x55b73512dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c760 tx=0x7f76c806e430 comp rx=0 tx=0).stop 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- 172.21.15.49:0/511321565 shutdown_connections 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b73513d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735129150 0x55b73512dec0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 --2- 172.21.15.49:0/511321565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735128830 0x55b735131490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- 172.21.15.49:0/511321565 >> 172.21.15.49:0/511321565 conn(0x55b735122f60 msgr2=0x55b735123340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- 172.21.15.49:0/511321565 shutdown_connections 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- 172.21.15.49:0/511321565 wait complete. 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 -- start start 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.009+0000 7f76dc18a740 1 Processor -- start 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735129150 0x55b7351352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735135800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41abb80 con 0x55b735128830 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41acff0 con 0x55b735129150 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ac250 con 0x55b735131d30 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 10 client.0 _ll_register_callbacks cb 0x7f76cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49046/0 (socket says 172.21.15.49:49046) 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 -- 172.21.15.49:0/1489336160 learned_addr learned my addr 172.21.15.49:0/1489336160 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d37fe640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d27fc640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735129150 0x55b7351352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 -- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 msgr2=0x55b735135800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735135800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 -- 172.21.15.49:0/1489336160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735129150 msgr2=0x55b7351352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735129150 0x55b7351352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 -- 172.21.15.49:0/1489336160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b735135d40 con 0x55b735128830 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76d2ffd640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f76cc003f60 tx=0x7f76cc0054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f121adf427afb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc02bc30 con 0x55b735128830 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f76cc02bdd0 con 0x55b735128830 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.010+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c809b750 con 0x55b735128830 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f76cc039050 con 0x55b735128830 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 10 client.17649 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:45.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b7351a8140 con 0x55b735128830 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c809b750 con 0x55b735128830 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f76c4032d10 con 0x55b735128830 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 10 client.17649 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 10 client.17649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 20 client.17649 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351a8720 0x55b7351aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 4 client.17649 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b735093d20 con 0x55b7351a8720 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.011+0000 7f76d27fc640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351a8720 0x55b7351aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76b6ff5640 10 client.17649 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d27fc640 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351a8720 0x55b7351aab10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e6f477efdc16b0d server_cookie=aa3a38ac7fd7632a in_seq=0 out_seq=0 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76b6ff5640 1 -- 172.21.15.49:0/1489336160 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b735093d20 con 0x55b7351a8720 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76b6ff5640 10 client.17649 handle_command_reply: tid=0 2024-03-16T08:25:45.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 client.17649 shutdown 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 msgr2=0x55b735139f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f76cc003f60 tx=0x7f76cc0054a0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 shutdown_connections 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55b7351a8720 0x55b7351aab10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b735131d30 0x55b735135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b735129150 0x55b7351352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76b6ff5640 0 client.17649 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 --2- 172.21.15.49:0/1489336160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b735128830 0x55b735139f70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 >> 172.21.15.49:0/1489336160 conn(0x55b735122f60 msgr2=0x55b7351243b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 shutdown_connections 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 1 -- 172.21.15.49:0/1489336160 wait complete. 2024-03-16T08:25:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76dc18a740 20 client.17649 trim_cache size 0 max 16384 2024-03-16T08:25:45.021 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:45.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 msgr2=0x7f76bc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 0x7f76bc0a4e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005e30 tx=0x7f76c402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 msgr2=0x7f76d4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f76c800ea60 tx=0x7f76c800ef20 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 shutdown_connections 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f76bc0a2970 0x7f76bc0a4e30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.012+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4107830 0x7f76d419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.013+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d40693a0 0x7f76d4198560 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.013+0000 7f76d95ab640 1 --2- 172.21.15.49:0/3074536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d40689e0 0x7f76d4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.013+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 >> 172.21.15.49:0/3074536735 conn(0x7f76d40777e0 msgr2=0x7f76d40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.013+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 shutdown_connections 2024-03-16T08:25:45.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.013+0000 7f76d95ab640 1 -- 172.21.15.49:0/3074536735 wait complete. 2024-03-16T08:25:45.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 -- 172.21.15.49:0/1506041186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 msgr2=0x7f3afc10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/1506041186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc10e650 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009cd0 tx=0x7f3af402c0f0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 -- 172.21.15.49:0/1506041186 shutdown_connections 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/1506041186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc10e650 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/1506041186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 0x7f3afc101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/1506041186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 0x7f3afc107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 -- 172.21.15.49:0/1506041186 >> 172.21.15.49:0/1506041186 conn(0x7f3afc0fd210 msgr2=0x7f3afc0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 -- 172.21.15.49:0/1506041186 shutdown_connections 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.133+0000 7f3b00cf8640 1 -- 172.21.15.49:0/1506041186 wait complete. 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 Processor -- start 2024-03-16T08:25:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 -- start start 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 0x7f3afc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 0x7f3afc19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111ba0 con 0x7f3afc102140 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111a20 con 0x7f3afc107790 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3b00cf8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111d20 con 0x7f3afc101780 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49064/0 (socket says 172.21.15.49:49064) 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.134+0000 7f3afa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 0x7f3afc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 -- 172.21.15.49:0/3835560560 learned_addr learned my addr 172.21.15.49:0/3835560560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3afad9d640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 0x7f3afc19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 -- 172.21.15.49:0/3835560560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 msgr2=0x7f3afc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 0x7f3afc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 -- 172.21.15.49:0/3835560560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 msgr2=0x7f3afc19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 0x7f3afc19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 -- 172.21.15.49:0/3835560560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc19d050 con 0x7f3afc102140 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400ea60 tx=0x7f3ae400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af37fe640 1 -- 172.21.15.49:0/3835560560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400cde0 con 0x7f3afc102140 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af37fe640 1 -- 172.21.15.49:0/3835560560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae40035f0 con 0x7f3afc102140 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3af37fe640 1 -- 172.21.15.49:0/3835560560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4010670 con 0x7f3afc102140 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc19d340 con 0x7f3afc102140 2024-03-16T08:25:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.135+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc1a4bd0 con 0x7f3afc102140 2024-03-16T08:25:45.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.137+0000 7f3af37fe640 1 -- 172.21.15.49:0/3835560560 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3ae40037a0 con 0x7f3afc102140 2024-03-16T08:25:45.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.137+0000 7f3b039c2740 1 Processor -- start 2024-03-16T08:25:45.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 -- start start 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d1764e0 0x55e66d1768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d183c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d186530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e66d0da490 con 0x55e66d184140 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e66d082c90 con 0x55e66d1764e0 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3b039c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e66d0d6530 con 0x55e66d176e00 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3af37fe640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 0x7f3ae80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d186530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d183c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d186530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49072/0 (socket says 172.21.15.49:49072) 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3af3fff640 1 -- 172.21.15.49:0/2930320061 learned_addr learned my addr 172.21.15.49:0/2930320061 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afad9d640 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d1764e0 0x55e66d1768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afad9d640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 0x7f3ae80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3af37fe640 1 -- 172.21.15.49:0/3835560560 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f3ae40cf0c0 con 0x7f3afc102140 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 -- 172.21.15.49:0/2930320061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d1764e0 msgr2=0x55e66d1768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d1764e0 0x55e66d1768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 -- 172.21.15.49:0/2930320061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 msgr2=0x55e66d186530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d186530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.138+0000 7f3afa59c640 1 -- 172.21.15.49:0/2930320061 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66d13d500 con 0x55e66d176e00 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.139+0000 7f3afa59c640 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d183c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009c20 tx=0x7f3aec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51ca14ca0d4204a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.139+0000 7f3afad9d640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 0x7f3ae80a4eb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005cc0 tx=0x7f3af4005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.139+0000 7f3af0ff9640 1 -- 172.21.15.49:0/2930320061 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004050 con 0x55e66d176e00 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.139+0000 7f3af0ff9640 1 -- 172.21.15.49:0/2930320061 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005500 con 0x55e66d176e00 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.139+0000 7f3af0ff9640 1 -- 172.21.15.49:0/2930320061 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004d60 con 0x55e66d176e00 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- 172.21.15.49:0/2930320061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 msgr2=0x55e66d183c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d183c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009c20 tx=0x7f3aec011810 comp rx=0 tx=0).stop 2024-03-16T08:25:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- 172.21.15.49:0/2930320061 shutdown_connections 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d186530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d183c00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- 172.21.15.49:0/2930320061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d1764e0 0x55e66d1768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- 172.21.15.49:0/2930320061 >> 172.21.15.49:0/2930320061 conn(0x55e66d170f60 msgr2=0x55e66d171340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- 172.21.15.49:0/2930320061 shutdown_connections 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- 172.21.15.49:0/2930320061 wait complete. 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- start start 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 Processor -- start 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d181650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d184140 0x55e66d181b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e66d186f80 con 0x55e66d1764e0 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e66d096230 con 0x55e66d176e00 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e66d187100 con 0x55e66d184140 2024-03-16T08:25:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3afa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.140+0000 7f3b039c2740 10 client.0 _ll_register_callbacks cb 0x55e66d0d2010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49088/0 (socket says 172.21.15.49:49088) 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 -- 172.21.15.49:0/1205308155 learned_addr learned my addr 172.21.15.49:0/1205308155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3af3fff640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d181650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afad9d640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d184140 0x55e66d181b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 -- 172.21.15.49:0/1205308155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d184140 msgr2=0x55e66d181b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d184140 0x55e66d181b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 -- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 msgr2=0x55e66d181650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d181650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 -- 172.21.15.49:0/1205308155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66d0a5100 con 0x55e66d1764e0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3afa59c640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3aec000c00 tx=0x7f3aec004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb5d5c1b8f5b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec029070 con 0x55e66d1764e0 2024-03-16T08:25:45.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f3aec0037f0 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec005500 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3aec035050 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3b039c2740 10 client.17661 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.141+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e66cdee3f0 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005500 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3aec0051c0 con 0x55e66d1764e0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 10 client.17661 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 20 client.17661 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1943b0 0x55e66d1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 4 client.17661 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e66d0e79d0 con 0x55e66d1943b0 2024-03-16T08:25:45.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3af3fff640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1943b0 0x55e66d1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3ae1ffb640 10 client.17661 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.142+0000 7f3af3fff640 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1943b0 0x55e66d1967a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e46a24be57a3bdd server_cookie=ff84d60ad5f1586a in_seq=0 out_seq=0 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/1205308155 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e66d0e79d0 con 0x55e66d1943b0 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3ae1ffb640 10 client.17661 handle_command_reply: tid=0 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 client.17661 shutdown 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 msgr2=0x55e66d181110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3aec000c00 tx=0x7f3aec004540 comp rx=0 tx=0).stop 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 shutdown_connections 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1943b0 0x55e66d1967a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d184140 0x55e66d181b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d181650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3ae1ffb640 0 client.17661 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 --2- 172.21.15.49:0/1205308155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d181110 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 >> 172.21.15.49:0/1205308155 conn(0x55e66d170f60 msgr2=0x55e66d172cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.143+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 shutdown_connections 2024-03-16T08:25:45.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.144+0000 7f3b039c2740 1 -- 172.21.15.49:0/1205308155 wait complete. 2024-03-16T08:25:45.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.144+0000 7f3b039c2740 20 client.17661 trim_cache size 0 max 16384 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 Processor -- start 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 -- start start 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d1764e0 0x55e66d183840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d1809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1adc10 con 0x55e66d184140 2024-03-16T08:25:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ab4e0 con 0x55e66d176e00 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3b039c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1a9350 con 0x55e66d1764e0 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49096/0 (socket says 172.21.15.49:49096) 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 -- 172.21.15.49:0/93919286 learned_addr learned my addr 172.21.15.49:0/93919286 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afa59c640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d1764e0 0x55e66d183840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3af3fff640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d1809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 -- 172.21.15.49:0/93919286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d1764e0 msgr2=0x55e66d183840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d1764e0 0x55e66d183840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 -- 172.21.15.49:0/93919286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 msgr2=0x55e66d1809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.157+0000 7f3afad9d640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d1809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.158+0000 7f3afad9d640 1 -- 172.21.15.49:0/93919286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66d196b10 con 0x55e66d184140 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.158+0000 7f3af3fff640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d1809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:45.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.158+0000 7f3afad9d640 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3af4036000 tx=0x7f3af402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa64ae0cc62d7115 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.160+0000 7f3ae3fff640 1 -- 172.21.15.49:0/93919286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4039040 con 0x55e66d184140 2024-03-16T08:25:45.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.160+0000 7f3ae3fff640 1 -- 172.21.15.49:0/93919286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af4055070 con 0x55e66d184140 2024-03-16T08:25:45.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.160+0000 7f3ae3fff640 1 -- 172.21.15.49:0/93919286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4050460 con 0x55e66d184140 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- 172.21.15.49:0/93919286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 msgr2=0x55e66d18fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3af4036000 tx=0x7f3af402e040 comp rx=0 tx=0).stop 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- 172.21.15.49:0/93919286 shutdown_connections 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d184140 0x55e66d18fd90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d176e00 0x55e66d1809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 --2- 172.21.15.49:0/93919286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d1764e0 0x55e66d183840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- 172.21.15.49:0/93919286 >> 172.21.15.49:0/93919286 conn(0x55e66d170f60 msgr2=0x55e66d16ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- 172.21.15.49:0/93919286 shutdown_connections 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- 172.21.15.49:0/93919286 wait complete. 2024-03-16T08:25:45.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 -- start start 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.161+0000 7f3b039c2740 1 Processor -- start 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d18c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d184140 0x55e66d187cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1a99a0 con 0x55e66d1764e0 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1aae10 con 0x55e66d184140 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1aa070 con 0x55e66d176e00 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3b039c2740 10 client.0 _ll_register_callbacks cb 0x7f3ae40972c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3afa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d18c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:42834/0 (socket says 172.21.15.49:42834) 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 -- 172.21.15.49:0/3751736150 learned_addr learned my addr 172.21.15.49:0/3751736150 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3afad9d640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d184140 0x55e66d187cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 -- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d184140 msgr2=0x55e66d187cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d184140 0x55e66d187cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 -- 172.21.15.49:0/3751736150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 msgr2=0x55e66d18c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d18c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 -- 172.21.15.49:0/3751736150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66ceb87d0 con 0x55e66d176e00 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3afa59c640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d18c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3af3fff640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40cce30 tx=0x7f3ae40979e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b716f249aaa7de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:45.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.162+0000 7f3ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:25:45.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae409b740 con 0x55e66d176e00 2024-03-16T08:25:45.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7f3ae4003b30 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af4055070 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ae40ccab0 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 10 client.27550 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e66d1fa150 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af4055070 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3ae40d10e0 con 0x55e66d176e00 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 10 client.27550 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 10 client.27550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 20 client.27550 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1fa820 0x55e66d1fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 4 client.27550 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.165+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e66d0e1d20 con 0x55e66d1fa820 2024-03-16T08:25:45.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3afa59c640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1fa820 0x55e66d1fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3afa59c640 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1fa820 0x55e66d1fcc40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f600c211ff8b1dad server_cookie=d9c4d0c129bacc4d in_seq=0 out_seq=0 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3ae1ffb640 10 client.27550 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3ae1ffb640 1 -- 172.21.15.49:0/3751736150 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e66d0e1d20 con 0x55e66d1fa820 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3ae1ffb640 10 client.27550 handle_command_reply: tid=0 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 client.27550 shutdown 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 msgr2=0x55e66d1877b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40cce30 tx=0x7f3ae40979e0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 shutdown_connections 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e66d1fa820 0x55e66d1fcc40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e66d184140 0x55e66d187cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e66d176e00 0x55e66d1877b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 --2- 172.21.15.49:0/3751736150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e66d1764e0 0x55e66d18c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3ae1ffb640 0 client.27550 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:45.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 >> 172.21.15.49:0/3751736150 conn(0x55e66d170f60 msgr2=0x55e66d181060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 shutdown_connections 2024-03-16T08:25:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 1 -- 172.21.15.49:0/3751736150 wait complete. 2024-03-16T08:25:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.166+0000 7f3b039c2740 20 client.27550 trim_cache size 0 max 16384 2024-03-16T08:25:45.171 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 msgr2=0x7f3ae80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 0x7f3ae80a4eb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005cc0 tx=0x7f3af4005c50 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 msgr2=0x7f3afc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400ea60 tx=0x7f3ae400ef20 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 shutdown_connections 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ae80a29f0 0x7f3ae80a4eb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3afc107790 0x7f3afc19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3afc102140 0x7f3afc1985a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 --2- 172.21.15.49:0/3835560560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc101780 0x7f3afc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 >> 172.21.15.49:0/3835560560 conn(0x7f3afc0fd210 msgr2=0x7f3afc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 shutdown_connections 2024-03-16T08:25:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:45.167+0000 7f3b00cf8640 1 -- 172.21.15.49:0/3835560560 wait complete. 2024-03-16T08:25:45.287 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:45 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3331902834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:45.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:45 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3331902834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:45.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:45 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3331902834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:25:46.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:25:46.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:25:46.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:25:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.304+0000 7fdbe9929640 1 -- 172.21.15.49:0/3704172621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe4106710 msgr2=0x7fdbe4106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.304+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3704172621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe4106710 0x7fdbe4106af0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009930 tx=0x7fdbd402bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 -- 172.21.15.49:0/3704172621 shutdown_connections 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3704172621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe41010c0 0x7fdbe410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3704172621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4100700 0x7fdbe4100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3704172621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe4106710 0x7fdbe4106af0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 -- 172.21.15.49:0/3704172621 >> 172.21.15.49:0/3704172621 conn(0x7fdbe4077840 msgr2=0x7fdbe4077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 -- 172.21.15.49:0/3704172621 shutdown_connections 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 -- 172.21.15.49:0/3704172621 wait complete. 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 Processor -- start 2024-03-16T08:25:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 -- start start 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.305+0000 7fdbe9929640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4106710 0x7fdbe40fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4110b20 con 0x7fdbe41010c0 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41109a0 con 0x7fdbe4100700 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4110ca0 con 0x7fdbe4106710 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbdbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:53720/0 (socket says 172.21.15.49:53720) 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbdbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49140/0 (socket says 172.21.15.49:49140) 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3055108743 learned_addr learned my addr 172.21.15.49:0/3055108743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4106710 0x7fdbe40fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3055108743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4106710 msgr2=0x7fdbe40fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4106710 0x7fdbe40fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 msgr2=0x7fdbe40fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3055108743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe41a6630 con 0x7fdbe4100700 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009230 tx=0x7fdbd402cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe8927640 1 -- 172.21.15.49:0/3055108743 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4041070 con 0x7fdbe4100700 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.306+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe41a68c0 con 0x7fdbe4100700 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.307+0000 7fdbe8927640 1 -- 172.21.15.49:0/3055108743 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd4004450 con 0x7fdbe4100700 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.307+0000 7fdbe8927640 1 -- 172.21.15.49:0/3055108743 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4003de0 con 0x7fdbe4100700 2024-03-16T08:25:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.307+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41a6db0 con 0x7fdbe4100700 2024-03-16T08:25:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbe8927640 1 -- 172.21.15.49:0/3055108743 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fdbd4044070 con 0x7fdbe4100700 2024-03-16T08:25:46.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 Processor -- start 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 -- start start 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc560330d0 con 0x55dc560cfa30 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc55fdbc70 con 0x55dc560d8da0 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc5602f170 con 0x55dc560d0710 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.309+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49152/0 (socket says 172.21.15.49:49152) 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3409766554 learned_addr learned my addr 172.21.15.49:0/3409766554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe8927640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 0x7fdbd00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3409766554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 msgr2=0x55dc560cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3409766554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 msgr2=0x55dc560db190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560db190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/3409766554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc55e0f7e0 con 0x55dc560cfa30 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 0x7fdbd00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe8927640 1 -- 172.21.15.49:0/3055108743 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fdbd40f06e0 con 0x7fdbe4100700 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40ede40 tx=0x7fdbd40b9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcf95efb7be4954 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbd9ffb640 1 -- 172.21.15.49:0/3409766554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40b90b0 con 0x55dc560cfa30 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbd9ffb640 1 -- 172.21.15.49:0/3409766554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40bd510 con 0x55dc560cfa30 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.310+0000 7fdbd9ffb640 1 -- 172.21.15.49:0/3409766554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40bc770 con 0x55dc560cfa30 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- 172.21.15.49:0/3409766554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 msgr2=0x55dc560cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 0x7fdbd00a4e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005fd0 tx=0x7fdbdc014040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40ede40 tx=0x7fdbd40b9750 comp rx=0 tx=0).stop 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- 172.21.15.49:0/3409766554 shutdown_connections 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560cfeb0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/3409766554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- 172.21.15.49:0/3409766554 >> 172.21.15.49:0/3409766554 conn(0x55dc56096c50 msgr2=0x55dc560ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- 172.21.15.49:0/3409766554 shutdown_connections 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- 172.21.15.49:0/3409766554 wait complete. 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.311+0000 7fdbec5f2740 1 -- start start 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 Processor -- start 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 0x55dc560d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc560db9d0 con 0x55dc560d8da0 2024-03-16T08:25:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc55fef0d0 con 0x55dc560cfa30 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc560dbb50 con 0x55dc560d0710 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbec5f2740 10 client.0 _ll_register_callbacks cb 0x55dc55fb8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 0x55dc560d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49162/0 (socket says 172.21.15.49:49162) 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 -- 172.21.15.49:0/2284757614 learned_addr learned my addr 172.21.15.49:0/2284757614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 -- 172.21.15.49:0/2284757614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 msgr2=0x55dc560d6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 -- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 msgr2=0x55dc560d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 0x55dc560d5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe37fe640 1 -- 172.21.15.49:0/2284757614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc55e6ff70 con 0x55dc560d8da0 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.312+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 0x55dc560d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc010c50 tx=0x7fdbdc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fae722a8fe18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc01eba0 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fdbdc01ed40 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd40bd510 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc01c480 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbec5f2740 10 client.17679 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc55d07170 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40bd510 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdbdc02e020 con 0x55dc560d8da0 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbec5f2740 10 client.17679 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.313+0000 7fdbec5f2740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbec5f2740 20 client.17679 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc560e8b80 0x55dc560eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbec5f2740 4 client.17679 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc56040610 con 0x55dc560e8b80 2024-03-16T08:25:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc560e8b80 0x55dc560eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc560e8b80 0x55dc560eafe0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d366e395c236f4b2 server_cookie=a68bf047c06ac331 in_seq=0 out_seq=0 2024-03-16T08:25:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.314+0000 7fdbc2ffd640 10 client.17679 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/2284757614 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dc56040610 con 0x55dc560e8b80 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbc2ffd640 10 client.17679 handle_command_reply: tid=0 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 client.17679 shutdown 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 msgr2=0x55dc560d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc010c50 tx=0x7fdbdc009500 comp rx=0 tx=0).stop 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 shutdown_connections 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc560e8b80 0x55dc560eafe0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560d8da0 0x55dc560d6760 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbc2ffd640 0 client.17679 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/2284757614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560cfa30 0x55dc560d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 >> 172.21.15.49:0/2284757614 conn(0x55dc56096c50 msgr2=0x55dc560cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 shutdown_connections 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 1 -- 172.21.15.49:0/2284757614 wait complete. 2024-03-16T08:25:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.315+0000 7fdbec5f2740 20 client.17679 trim_cache size 0 max 16384 2024-03-16T08:25:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 Processor -- start 2024-03-16T08:25:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 -- start start 2024-03-16T08:25:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41b1e90 con 0x55dc560cfa30 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41af760 con 0x55dc560d8da0 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.333+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41ad5d0 con 0x55dc560d0710 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49174/0 (socket says 172.21.15.49:49174) 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/218323519 learned_addr learned my addr 172.21.15.49:0/218323519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/218323519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 msgr2=0x55dc560d5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/218323519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 msgr2=0x55dc560f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/218323519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc560cea60 con 0x55dc560cfa30 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdbd402cfd0 tx=0x7fdbd4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1fa306a5937ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbd97fa640 1 -- 172.21.15.49:0/218323519 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd404b040 con 0x55dc560cfa30 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbd97fa640 1 -- 172.21.15.49:0/218323519 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40bd510 con 0x55dc560cfa30 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.334+0000 7fdbd97fa640 1 -- 172.21.15.49:0/218323519 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40bc950 con 0x55dc560cfa30 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- 172.21.15.49:0/218323519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 msgr2=0x55dc560e4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdbd402cfd0 tx=0x7fdbd4000bc0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- 172.21.15.49:0/218323519 shutdown_connections 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d8da0 0x55dc560f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d0710 0x55dc560d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/218323519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560e4770 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- 172.21.15.49:0/218323519 >> 172.21.15.49:0/218323519 conn(0x55dc56096c50 msgr2=0x55dc560d7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- 172.21.15.49:0/218323519 shutdown_connections 2024-03-16T08:25:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- 172.21.15.49:0/218323519 wait complete. 2024-03-16T08:25:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 -- start start 2024-03-16T08:25:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.335+0000 7fdbec5f2740 1 Processor -- start 2024-03-16T08:25:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d0710 0x55dc560d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d8da0 0x55dc560d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41adc20 con 0x55dc560cfa30 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41af090 con 0x55dc560d0710 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41ae2f0 con 0x55dc560d8da0 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 10 client.0 _ll_register_callbacks cb 0x7fdbdc004720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49184/0 (socket says 172.21.15.49:49184) 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/822736484 learned_addr learned my addr 172.21.15.49:0/822736484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe37fe640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d8da0 0x55dc560d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d0710 0x55dc560d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/822736484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d8da0 msgr2=0x55dc560d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d8da0 0x55dc560d3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d0710 msgr2=0x55dc560d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d0710 0x55dc560d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 -- 172.21.15.49:0/822736484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5614e130 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbe2ffd640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40bcb10 tx=0x7fdbd4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f164231d84ee08c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40bcbe0 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fdbd40035f0 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.336+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd40bd510 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdbd40bc950 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 10 client.17685 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dc5614f430 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40bd510 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdbd40c5120 con 0x55dc560cfa30 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 10 client.17685 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 20 client.17685 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc5614f770 0x55dc56151b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 4 client.17685 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc5603a960 con 0x55dc5614f770 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.337+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc5614f770 0x55dc56151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.338+0000 7fdbdbfff640 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc5614f770 0x55dc56151b60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ffd1369a051a49 server_cookie=c5e1db2ac6737540 in_seq=0 out_seq=0 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.338+0000 7fdbc2ffd640 10 client.17685 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbc2ffd640 1 -- 172.21.15.49:0/822736484 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc5603a960 con 0x55dc5614f770 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbc2ffd640 10 client.17685 handle_command_reply: tid=0 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 client.17685 shutdown 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 msgr2=0x55dc560d2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40bcb10 tx=0x7fdbd4005b50 comp rx=0 tx=0).stop 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 shutdown_connections 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55dc5614f770 0x55dc56151b60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc560d8da0 0x55dc560d3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbc2ffd640 0 client.17685 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc560d0710 0x55dc560d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 --2- 172.21.15.49:0/822736484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc560cfa30 0x55dc560d2b10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 >> 172.21.15.49:0/822736484 conn(0x55dc56096c50 msgr2=0x55dc560d6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 shutdown_connections 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.339+0000 7fdbec5f2740 1 -- 172.21.15.49:0/822736484 wait complete. 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbec5f2740 20 client.17685 trim_cache size 0 max 16384 2024-03-16T08:25:46.347 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:46.348 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:25:46.348 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "66d5cd33-7307-48bc-9d67-564b1df27360", 2024-03-16T08:25:46.348 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:25:46.348 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 msgr2=0x7fdbd00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 0x7fdbd00a4e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005fd0 tx=0x7fdbdc014040 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 msgr2=0x7fdbe40fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009230 tx=0x7fdbd402cb30 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 shutdown_connections 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fdbd00a2970 0x7fdbd00a4e30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbe4106710 0x7fdbe40fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe41010c0 0x7fdbe40fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 --2- 172.21.15.49:0/3055108743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4100700 0x7fdbe40fd680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 >> 172.21.15.49:0/3055108743 conn(0x7fdbe4077840 msgr2=0x7fdbe406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 shutdown_connections 2024-03-16T08:25:46.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.340+0000 7fdbe9929640 1 -- 172.21.15.49:0/3055108743 wait complete. 2024-03-16T08:25:46.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:25:46.352 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 -- 172.21.15.49:0/373295549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344106710 msgr2=0x7fd344106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 --2- 172.21.15.49:0/373295549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344106710 0x7fd344106af0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd334009930 tx=0x7fd33402bc30 comp rx=0 tx=0).stop 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 -- 172.21.15.49:0/373295549 shutdown_connections 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 --2- 172.21.15.49:0/373295549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3441010c0 0x7fd34410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 --2- 172.21.15.49:0/373295549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344100700 0x7fd344100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 --2- 172.21.15.49:0/373295549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344106710 0x7fd344106af0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 -- 172.21.15.49:0/373295549 >> 172.21.15.49:0/373295549 conn(0x7fd344077840 msgr2=0x7fd344077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 -- 172.21.15.49:0/373295549 shutdown_connections 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.461+0000 7fd348cf1640 1 -- 172.21.15.49:0/373295549 wait complete. 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 Processor -- start 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 -- start start 2024-03-16T08:25:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3441010c0 0x7fd344198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd344106710 0x7fd34419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd344110ae0 con 0x7fd344100700 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd344110960 con 0x7fd3441010c0 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd348cf1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd344110c60 con 0x7fd344106710 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49198/0 (socket says 172.21.15.49:49198) 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd34259c640 1 -- 172.21.15.49:0/3861369314 learned_addr learned my addr 172.21.15.49:0/3861369314 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.462+0000 7fd342d9d640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd344106710 0x7fd34419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd33bfff640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3441010c0 0x7fd344198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 -- 172.21.15.49:0/3861369314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd344106710 msgr2=0x7fd34419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd344106710 0x7fd34419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 -- 172.21.15.49:0/3861369314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3441010c0 msgr2=0x7fd344198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3441010c0 0x7fd344198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 -- 172.21.15.49:0/3861369314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34419d1c0 con 0x7fd344100700 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd34259c640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd334009230 tx=0x7fd33402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd33b7fe640 1 -- 172.21.15.49:0/3861369314 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334041070 con 0x7fd344100700 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34419d450 con 0x7fd344100700 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd33b7fe640 1 -- 172.21.15.49:0/3861369314 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd334004450 con 0x7fd344100700 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd33b7fe640 1 -- 172.21.15.49:0/3861369314 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334003e50 con 0x7fd344100700 2024-03-16T08:25:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.463+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441a4d40 con 0x7fd344100700 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.465+0000 7fd33b7fe640 1 -- 172.21.15.49:0/3861369314 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd334044070 con 0x7fd344100700 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 Processor -- start 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 -- start start 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd33b7fe640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 0x7fd3300a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561741db6500 con 0x561741e51b30 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561741d5ebe0 con 0x561741e5fa60 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561741db25a0 con 0x561741e524e0 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd33bfff640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 0x7fd3300a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49200/0 (socket says 172.21.15.49:49200) 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 -- 172.21.15.49:0/4270726928 learned_addr learned my addr 172.21.15.49:0/4270726928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd34259c640 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd33b7fe640 1 -- 172.21.15.49:0/3861369314 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fd3340f0600 con 0x7fd344100700 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 -- 172.21.15.49:0/4270726928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 msgr2=0x561741e5f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 -- 172.21.15.49:0/4270726928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 msgr2=0x561741e61e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.466+0000 7fd342d9d640 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e61e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd342d9d640 1 -- 172.21.15.49:0/4270726928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561741d724b0 con 0x561741e51b30 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd33bfff640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 0x7fd3300a4fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0095f0 tx=0x7fd32c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd342d9d640 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005c40 tx=0x7fd33c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2083394b0c409554 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd338ff9640 1 -- 172.21.15.49:0/4270726928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c013e60 con 0x561741e51b30 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd338ff9640 1 -- 172.21.15.49:0/4270726928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c00f8c0 con 0x561741e51b30 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.467+0000 7fd338ff9640 1 -- 172.21.15.49:0/4270726928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005530 con 0x561741e51b30 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- 172.21.15.49:0/4270726928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 msgr2=0x561741e51f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005c40 tx=0x7fd33c005bd0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- 172.21.15.49:0/4270726928 shutdown_connections 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/4270726928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e51f10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- 172.21.15.49:0/4270726928 >> 172.21.15.49:0/4270726928 conn(0x561741e4c880 msgr2=0x561741e4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- 172.21.15.49:0/4270726928 shutdown_connections 2024-03-16T08:25:46.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- 172.21.15.49:0/4270726928 wait complete. 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- start start 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 Processor -- start 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e5cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e5fa60 0x561741e5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561741e62820 con 0x561741e51b30 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561741d72160 con 0x561741e524e0 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.468+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561741e629a0 con 0x561741e5fa60 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34b9ba740 10 client.0 _ll_register_callbacks cb 0x561741d56150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49208/0 (socket says 172.21.15.49:49208) 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 -- 172.21.15.49:0/2992737291 learned_addr learned my addr 172.21.15.49:0/2992737291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd33bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e5cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd342d9d640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e5fa60 0x561741e5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 -- 172.21.15.49:0/2992737291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e5fa60 msgr2=0x561741e5d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e5fa60 0x561741e5d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 -- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 msgr2=0x561741e5cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e5cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 -- 172.21.15.49:0/2992737291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561741b947d0 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34259c640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd3340b92a0 tx=0x7fd3340bccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f98b3ecc638b8c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340d2070 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fd3340dadc0 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c00f8c0 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd33404b040 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34b9ba740 10 client.17697 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561741aca3f0 con 0x561741e51b30 2024-03-16T08:25:46.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.469+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3340d5e50 con 0x561741e51b30 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd3340d5020 con 0x561741e51b30 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 10 client.17697 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 20 client.17697 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741e6f9f0 0x561741e71e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 4 client.17697 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561741dc3a40 con 0x561741e6f9f0 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd33bfff640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741e6f9f0 0x561741e71e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd329ffb640 10 client.17697 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.470+0000 7fd33bfff640 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741e6f9f0 0x561741e71e60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47abdacda8a5b804 server_cookie=10308dc606959f7e in_seq=0 out_seq=0 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd329ffb640 1 -- 172.21.15.49:0/2992737291 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561741dc3a40 con 0x561741e6f9f0 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd329ffb640 10 client.17697 handle_command_reply: tid=0 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 client.17697 shutdown 2024-03-16T08:25:46.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 msgr2=0x561741e5ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd3340b92a0 tx=0x7fd3340bccd0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 shutdown_connections 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741e6f9f0 0x561741e71e60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e5fa60 0x561741e5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e5cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd329ffb640 0 client.17697 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2992737291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5ca60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 >> 172.21.15.49:0/2992737291 conn(0x561741e4c880 msgr2=0x561741e4e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 shutdown_connections 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2992737291 wait complete. 2024-03-16T08:25:46.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.471+0000 7fd34b9ba740 20 client.17697 trim_cache size 0 max 16384 2024-03-16T08:25:46.480 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:46 smithi154 ceph-mon[36471]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 Processor -- start 2024-03-16T08:25:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 -- start start 2024-03-16T08:25:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441ada20 con 0x561741e51b30 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441ab2f0 con 0x561741e5fa60 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441a9160 con 0x561741e524e0 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49214/0 (socket says 172.21.15.49:49214) 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 -- 172.21.15.49:0/3076638417 learned_addr learned my addr 172.21.15.49:0/3076638417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd33bfff640 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd342d9d640 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 -- 172.21.15.49:0/3076638417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 msgr2=0x561741e5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 -- 172.21.15.49:0/3076638417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 msgr2=0x561741e476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 -- 172.21.15.49:0/3076638417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561741a8a0f0 con 0x561741e51b30 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd34259c640 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd334004e00 tx=0x7fd3340bcd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11c542d9146501a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd32bfff640 1 -- 172.21.15.49:0/3076638417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334080070 con 0x561741e51b30 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd32bfff640 1 -- 172.21.15.49:0/3076638417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3340d5e50 con 0x561741e51b30 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.489+0000 7fd32bfff640 1 -- 172.21.15.49:0/3076638417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340d2040 con 0x561741e51b30 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- 172.21.15.49:0/3076638417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 msgr2=0x561741e5b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd334004e00 tx=0x7fd3340bcd40 comp rx=0 tx=0).stop 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- 172.21.15.49:0/3076638417 shutdown_connections 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e5fa60 0x561741e476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e524e0 0x561741e5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/3076638417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e51b30 0x561741e5b9a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- 172.21.15.49:0/3076638417 >> 172.21.15.49:0/3076638417 conn(0x561741e4c880 msgr2=0x561741e4e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- 172.21.15.49:0/3076638417 shutdown_connections 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- 172.21.15.49:0/3076638417 wait complete. 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 -- start start 2024-03-16T08:25:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.490+0000 7fd34b9ba740 1 Processor -- start 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e51b30 0x561741e63d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441a97b0 con 0x561741e5fa60 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441aac20 con 0x561741e524e0 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441a9e80 con 0x561741e51b30 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34b9ba740 10 client.0 _ll_register_callbacks cb 0x7fd334005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49228/0 (socket says 172.21.15.49:49228) 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd33bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 -- 172.21.15.49:0/2288497605 learned_addr learned my addr 172.21.15.49:0/2288497605 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd34259c640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e51b30 0x561741e63d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 -- 172.21.15.49:0/2288497605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e51b30 msgr2=0x561741e63d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e51b30 0x561741e63d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 -- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 msgr2=0x561741e642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 -- 172.21.15.49:0/2288497605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561741d7d9d0 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd33bfff640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd342d9d640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd33c01b000 tx=0x7fd33c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1532ce6b207a0cc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c013e60 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.491+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8492+0+0 (secure 0 0 0) 0x7fd33c003b70 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3340d5e50 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd33c030050 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 10 client.17703 fetch_fsmap learned FSMap version 24 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561741ed6460 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3340d5e50 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd33c01d070 con 0x561741e5fa60 2024-03-16T08:25:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 10 client.17703 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 20 client.17703 populate_metadata read hostname 'smithi049' 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741ed6a10 0x561741ed8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 4 client.17703 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561741dbdd90 con 0x561741ed6a10 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.492+0000 7fd34259c640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741ed6a10 0x561741ed8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34259c640 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741ed6a10 0x561741ed8e30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c443c0c4e73da01b server_cookie=ed8ab0d7a2992a96 in_seq=0 out_seq=0 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd329ffb640 10 client.17703 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd329ffb640 1 -- 172.21.15.49:0/2288497605 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561741dbdd90 con 0x561741ed6a10 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd329ffb640 10 client.17703 handle_command_reply: tid=0 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 client.17703 shutdown 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 msgr2=0x561741ecece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd33c01b000 tx=0x7fd33c00ff90 comp rx=0 tx=0).stop 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 shutdown_connections 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x561741ed6a10 0x561741ed8e30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561741e5fa60 0x561741ecece0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561741e524e0 0x561741e642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd329ffb640 0 client.17703 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 --2- 172.21.15.49:0/2288497605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561741e51b30 0x561741e63d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 >> 172.21.15.49:0/2288497605 conn(0x561741e4c880 msgr2=0x561741e4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 shutdown_connections 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 1 -- 172.21.15.49:0/2288497605 wait complete. 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd34b9ba740 20 client.17703 trim_cache size 0 max 16384 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:25:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: "66d5cd33-7307-48bc-9d67-564b1df27360": { 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "66d5cd33-7307-48bc-9d67-564b1df27360", 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:25:46.502 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 msgr2=0x7fd3300a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 0x7fd3300a4fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0095f0 tx=0x7fd32c005f90 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.493+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 msgr2=0x7fd3441981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd334009230 tx=0x7fd33402cb30 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 shutdown_connections 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd3300a2b10 0x7fd3300a4fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd344106710 0x7fd34419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3441010c0 0x7fd344198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 --2- 172.21.15.49:0/3861369314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd344100700 0x7fd3441981d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 >> 172.21.15.49:0/3861369314 conn(0x7fd344077840 msgr2=0x7fd34406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 shutdown_connections 2024-03-16T08:25:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:25:46.494+0000 7fd348cf1640 1 -- 172.21.15.49:0/3861369314 wait complete. 2024-03-16T08:25:46.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:46 smithi049 ceph-mon[31942]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:46.505 INFO:tasks.cephfs.filesystem:scrub status for tag:66d5cd33-7307-48bc-9d67-564b1df27360 - {'path': '/', 'tag': '66d5cd33-7307-48bc-9d67-564b1df27360', 'options': 'recursive,force'} 2024-03-16T08:25:46.511 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:46 smithi129 ceph-mon[36561]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:25:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:47 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:25:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:47 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:47 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:25:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:47 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:47 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:25:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:47 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:25:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:47 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:47 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:25:47.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:47 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:47 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:25:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:47 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:25:47.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:47 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:47 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:25:47.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:47 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:25:47.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:47 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:25:48.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:48 smithi154 ceph-mon[36471]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:48.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:48 smithi049 ceph-mon[31942]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:48.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:48 smithi129 ceph-mon[36561]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:25:50.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:50 smithi154 ceph-mon[36471]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:50.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:50 smithi049 ceph-mon[31942]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:50.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:50 smithi129 ceph-mon[36561]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:52.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:52 smithi154 ceph-mon[36471]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:52.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:52 smithi049 ceph-mon[31942]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:52.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:52 smithi129 ceph-mon[36561]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:54.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:54 smithi154 ceph-mon[36471]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:54.540 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:54 smithi049 ceph-mon[31942]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:54.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:54 smithi129 ceph-mon[36561]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:56.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:56 smithi154 ceph-mon[36471]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:56.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:56 smithi049 ceph-mon[31942]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:56.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:56 smithi129 ceph-mon[36561]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:25:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:25:58 smithi154 ceph-mon[36471]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:25:58 smithi049 ceph-mon[31942]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:25:58.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:25:58 smithi129 ceph-mon[36561]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:00 smithi154 ceph-mon[36471]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:00.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:00 smithi049 ceph-mon[31942]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:00.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:00 smithi129 ceph-mon[36561]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:02 smithi154 ceph-mon[36471]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:02 smithi049 ceph-mon[31942]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:02.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:02 smithi129 ceph-mon[36561]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:04 smithi154 ceph-mon[36471]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:04.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:04 smithi049 ceph-mon[31942]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:04.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:04 smithi129 ceph-mon[36561]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:06.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:06 smithi154 ceph-mon[36471]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:26:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:06 smithi049 ceph-mon[31942]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:26:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:06 smithi129 ceph-mon[36561]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-16T08:26:08.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:08 smithi154 ceph-mon[36471]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:26:08.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:08 smithi154 ceph-mon[36471]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T08:26:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:08 smithi049 ceph-mon[31942]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:26:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:08 smithi049 ceph-mon[31942]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T08:26:08.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:08 smithi129 ceph-mon[36561]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:26:08.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:08 smithi129 ceph-mon[36561]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T08:26:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:10 smithi154 ceph-mon[36471]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:10.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:10 smithi049 ceph-mon[31942]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:10.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:10 smithi129 ceph-mon[36561]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:12 smithi154 ceph-mon[36471]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:12.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:12 smithi049 ceph-mon[31942]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:12.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:12 smithi129 ceph-mon[36561]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:14.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:14 smithi154 ceph-mon[36471]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:14.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:14 smithi154 ceph-mon[36471]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T08:26:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:14 smithi049 ceph-mon[31942]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:14.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:14 smithi049 ceph-mon[31942]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T08:26:14.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:14 smithi129 ceph-mon[36561]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:26:14.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:14 smithi129 ceph-mon[36561]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T08:26:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:16 smithi154 ceph-mon[36471]: pgmap v280: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:26:16.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:26:16.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:26:16.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:16 smithi049 ceph-mon[31942]: pgmap v280: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:26:16.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:16 smithi129 ceph-mon[36561]: pgmap v280: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:26:16.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- 172.21.15.49:0/2039258759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa81010c0 msgr2=0x7f1aa810d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/2039258759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa81010c0 0x7f1aa810d5d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009cd0 tx=0x7f1aa002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- 172.21.15.49:0/2039258759 shutdown_connections 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/2039258759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa81010c0 0x7f1aa810d5d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/2039258759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/2039258759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa8106710 0x7f1aa8106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- 172.21.15.49:0/2039258759 >> 172.21.15.49:0/2039258759 conn(0x7f1aa8077840 msgr2=0x7f1aa8077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- 172.21.15.49:0/2039258759 shutdown_connections 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- 172.21.15.49:0/2039258759 wait complete. 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 Processor -- start 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.630+0000 7f1aaeff4640 1 -- start start 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa81010c0 0x7f1aa81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8110b20 con 0x7f1aa8106710 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81109a0 con 0x7f1aa8100700 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aaeff4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8110ca0 con 0x7f1aa81010c0 2024-03-16T08:26:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aacd90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58474/0 (socket says 172.21.15.49:58474) 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 -- 172.21.15.49:0/3449059404 learned_addr learned my addr 172.21.15.49:0/3449059404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa81010c0 0x7f1aa81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 -- 172.21.15.49:0/3449059404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa81010c0 msgr2=0x7f1aa81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa81010c0 0x7f1aa81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 -- 172.21.15.49:0/3449059404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 msgr2=0x7f1aa8198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 -- 172.21.15.49:0/3449059404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa819d160 con 0x7f1aa8106710 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aacd90640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aad591640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00348f0 tx=0x7f1aa0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aa5ffb640 1 -- 172.21.15.49:0/3449059404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0041020 con 0x7f1aa8106710 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.631+0000 7f1aa5ffb640 1 -- 172.21.15.49:0/3449059404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0032e80 con 0x7f1aa8106710 2024-03-16T08:26:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.632+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa819d3f0 con 0x7f1aa8106710 2024-03-16T08:26:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.632+0000 7f1aa5ffb640 1 -- 172.21.15.49:0/3449059404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0039040 con 0x7f1aa8106710 2024-03-16T08:26:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.632+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a4ce0 con 0x7f1aa8106710 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1aa5ffb640 1 -- 172.21.15.49:0/3449059404 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f1aa0044020 con 0x7f1aa8106710 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 Processor -- start 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 -- start start 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f6524a500 con 0x556f652ea560 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f651f2be0 con 0x556f652f3a60 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.634+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556f652465a0 con 0x556f652eae80 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652ea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58488/0 (socket says 172.21.15.49:58488) 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa5ffb640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 0x7f1a900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa7fff640 1 -- 172.21.15.49:0/2923862065 learned_addr learned my addr 172.21.15.49:0/2923862065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aad591640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 0x7f1a900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa5ffb640 1 -- 172.21.15.49:0/3449059404 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f1aa00f08e0 con 0x7f1aa8106710 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 -- 172.21.15.49:0/2923862065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 msgr2=0x556f652f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 -- 172.21.15.49:0/2923862065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 msgr2=0x556f652ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 -- 172.21.15.49:0/2923862065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f652064b0 con 0x556f652eae80 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f3520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009f50 tx=0x7f1a980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fbbd53e2a6cbce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1aad591640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 0x7f1a900a4e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00ee210 tx=0x7f1aa00095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1a8effd640 1 -- 172.21.15.49:0/2923862065 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98004030 con 0x556f652eae80 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1a8effd640 1 -- 172.21.15.49:0/2923862065 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a98005500 con 0x556f652eae80 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.635+0000 7f1a8effd640 1 -- 172.21.15.49:0/2923862065 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98004d50 con 0x556f652eae80 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2923862065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 msgr2=0x556f652f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f3520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009f50 tx=0x7f1a980118d0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2923862065 shutdown_connections 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f3520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2923862065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2923862065 >> 172.21.15.49:0/2923862065 conn(0x556f652e4c90 msgr2=0x556f652e5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2923862065 shutdown_connections 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.636+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2923862065 wait complete. 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 -- start start 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 Processor -- start 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f652f6850 con 0x556f652ea560 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f65206160 con 0x556f652f3a60 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556f652f69d0 con 0x556f652eae80 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1ab1be3740 10 client.0 _ll_register_callbacks cb 0x556f651ea150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58490/0 (socket says 172.21.15.49:58490) 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1aa7fff640 1 -- 172.21.15.49:0/3049458046 learned_addr learned my addr 172.21.15.49:0/3049458046 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1aacd90640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.637+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 -- 172.21.15.49:0/3049458046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 msgr2=0x556f652f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 -- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 msgr2=0x556f652f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 -- 172.21.15.49:0/3049458046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f650287d0 con 0x556f652ea560 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aacd90640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c012e70 tx=0x7f1a9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742fa9d7f9b25fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c01c040 con 0x556f652ea560 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f1a9c004030 con 0x556f652ea560 2024-03-16T08:26:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a98005500 con 0x556f652ea560 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c004230 con 0x556f652ea560 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1ab1be3740 10 client.17715 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f64f5e3f0 con 0x556f652ea560 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.638+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a98005500 con 0x556f652ea560 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a9c02a020 con 0x556f652ea560 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 10 client.17715 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 20 client.17715 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65303a60 0x556f65305e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 4 client.17715 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f65257a40 con 0x556f65303a60 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1aacd90640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65303a60 0x556f65305e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1aacd90640 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65303a60 0x556f65305e90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a2b4cc438f2ce server_cookie=4adc49075d0d460e in_seq=0 out_seq=0 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.639+0000 7f1a87fff640 10 client.17715 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1a87fff640 1 -- 172.21.15.49:0/3049458046 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556f65257a40 con 0x556f65303a60 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1a87fff640 10 client.17715 handle_command_reply: tid=0 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 client.17715 shutdown 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 msgr2=0x556f652f0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c012e70 tx=0x7f1a9c009780 comp rx=0 tx=0).stop 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 shutdown_connections 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65303a60 0x556f65305e90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652eae80 0x556f652f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1a87fff640 0 client.17715 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/3049458046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652ea560 0x556f652f0ac0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 >> 172.21.15.49:0/3049458046 conn(0x556f652e4c90 msgr2=0x556f652e69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 shutdown_connections 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 1 -- 172.21.15.49:0/3049458046 wait complete. 2024-03-16T08:26:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.640+0000 7f1ab1be3740 20 client.17715 trim_cache size 0 max 16384 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 Processor -- start 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 -- start start 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81adb10 con 0x556f652eae80 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81ab3e0 con 0x556f652f3a60 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a9250 con 0x556f652ea560 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1aacd90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1aacd90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58500/0 (socket says 172.21.15.49:58500) 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1aacd90640 1 -- 172.21.15.49:0/2268618717 learned_addr learned my addr 172.21.15.49:0/2268618717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.658+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 -- 172.21.15.49:0/2268618717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 msgr2=0x556f652f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 -- 172.21.15.49:0/2268618717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 msgr2=0x556f652df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 -- 172.21.15.49:0/2268618717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f652eacb0 con 0x556f652eae80 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1a98012fd0 tx=0x7f1a980096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bb84645af783d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1a8e7fc640 1 -- 172.21.15.49:0/2268618717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98032070 con 0x556f652eae80 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1a8e7fc640 1 -- 172.21.15.49:0/2268618717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9802c310 con 0x556f652eae80 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1a8e7fc640 1 -- 172.21.15.49:0/2268618717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9802a070 con 0x556f652eae80 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2268618717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 msgr2=0x556f652ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.659+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1a98012fd0 tx=0x7f1a980096c0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2268618717 shutdown_connections 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652f3a60 0x556f652df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652eae80 0x556f652ff620 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2268618717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2268618717 >> 172.21.15.49:0/2268618717 conn(0x556f652e4c90 msgr2=0x556f652efa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2268618717 shutdown_connections 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2268618717 wait complete. 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- start start 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 Processor -- start 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652eae80 0x556f652ed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a98a0 con 0x556f652f3a60 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81aad10 con 0x556f652eae80 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a9f70 con 0x556f652ea560 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.660+0000 7f1ab1be3740 10 client.0 _ll_register_callbacks cb 0x7f1a9c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aacd90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652eae80 0x556f652ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58506/0 (socket says 172.21.15.49:58506) 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 -- 172.21.15.49:0/2330572231 learned_addr learned my addr 172.21.15.49:0/2330572231 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 -- 172.21.15.49:0/2330572231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 msgr2=0x556f652fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 -- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652eae80 msgr2=0x556f652ed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652eae80 0x556f652ed310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 -- 172.21.15.49:0/2330572231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f64f1e0f0 con 0x556f652f3a60 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1aa7fff640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c012e70 tx=0x7f1a9c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b45ad848a2219e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c012190 con 0x556f652f3a60 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f1a9c005030 con 0x556f652f3a60 2024-03-16T08:26:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9802c310 con 0x556f652f3a60 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c012330 con 0x556f652f3a60 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1ab1be3740 10 client.17721 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.661+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f652119d0 con 0x556f652f3a60 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9802c310 con 0x556f652f3a60 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a9c038070 con 0x556f652f3a60 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 10 client.17721 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 20 client.17721 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65369ea0 0x556f6536c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 4 client.17721 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f65251d90 con 0x556f65369ea0 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65369ea0 0x556f6536c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1a87fff640 10 client.17721 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1aacd90640 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65369ea0 0x556f6536c2c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa340db0d610024b server_cookie=74276dd8c09d8b1e in_seq=0 out_seq=0 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1a87fff640 1 -- 172.21.15.49:0/2330572231 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f65251d90 con 0x556f65369ea0 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1a87fff640 10 client.17721 handle_command_reply: tid=0 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.662+0000 7f1ab1be3740 1 client.17721 shutdown 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 msgr2=0x556f652ed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c012e70 tx=0x7f1a9c014040 comp rx=0 tx=0).stop 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 shutdown_connections 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x556f65369ea0 0x556f6536c2c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f652f3a60 0x556f652ed850 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1a87fff640 0 client.17721 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f652eae80 0x556f652ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 --2- 172.21.15.49:0/2330572231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556f652ea560 0x556f652fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 >> 172.21.15.49:0/2330572231 conn(0x556f652e4c90 msgr2=0x556f652f1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 shutdown_connections 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 1 -- 172.21.15.49:0/2330572231 wait complete. 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1ab1be3740 20 client.17721 trim_cache size 0 max 16384 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:26:16.670 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 msgr2=0x7f1a900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 0x7f1a900a4e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00ee210 tx=0x7f1aa00095c0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 msgr2=0x7f1aa819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00348f0 tx=0x7f1aa0032c90 comp rx=0 tx=0).stop 2024-03-16T08:26:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 shutdown_connections 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f1a900a2970 0x7f1a900a4e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa8106710 0x7f1aa819ca30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1aa81010c0 0x7f1aa81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 --2- 172.21.15.49:0/3449059404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa8100700 0x7f1aa8198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.663+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 >> 172.21.15.49:0/3449059404 conn(0x7f1aa8077840 msgr2=0x7f1aa806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.664+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 shutdown_connections 2024-03-16T08:26:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.664+0000 7f1aaeff4640 1 -- 172.21.15.49:0/3449059404 wait complete. 2024-03-16T08:26:16.675 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:26:16.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 -- 172.21.15.49:0/1244056627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d68069e90 msgr2=0x7f5d6806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/1244056627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d68069e90 0x7f5d6806a2f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009900 tx=0x7f5d5002bcd0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 -- 172.21.15.49:0/1244056627 shutdown_connections 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/1244056627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d6806a830 0x7f5d6810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/1244056627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d68069e90 0x7f5d6806a2f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/1244056627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d681077e0 0x7f5d68107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 -- 172.21.15.49:0/1244056627 >> 172.21.15.49:0/1244056627 conn(0x7f5d68077780 msgr2=0x7f5d68077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 -- 172.21.15.49:0/1244056627 shutdown_connections 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.823+0000 7f5d6cb62640 1 -- 172.21.15.49:0/1244056627 wait complete. 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 Processor -- start 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 -- start start 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d68069e90 0x7f5d6819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 0x7f5d6819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68111ba0 con 0x7f5d6806a830 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68111a20 con 0x7f5d681077e0 2024-03-16T08:26:16.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.824+0000 7f5d6cb62640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68111d20 con 0x7f5d68069e90 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58518/0 (socket says 172.21.15.49:58518) 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 -- 172.21.15.49:0/4117044092 learned_addr learned my addr 172.21.15.49:0/4117044092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d66d9d640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 0x7f5d6819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d6659c640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d68069e90 0x7f5d6819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 -- 172.21.15.49:0/4117044092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d68069e90 msgr2=0x7f5d6819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d68069e90 0x7f5d6819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 -- 172.21.15.49:0/4117044092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 msgr2=0x7f5d6819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 0x7f5d6819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 -- 172.21.15.49:0/4117044092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6819f310 con 0x7f5d6806a830 2024-03-16T08:26:16.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d66d9d640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 0x7f5d6819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5ffff640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005940 tx=0x7f5d5002cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50041070 con 0x7f5d6806a830 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6819f5a0 con 0x7f5d6806a830 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d50004450 con 0x7f5d6806a830 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50003d30 con 0x7f5d6806a830 2024-03-16T08:26:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.825+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d681a6e90 con 0x7f5d6806a830 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.826+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d68102f00 con 0x7f5d6806a830 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.828+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5d50044070 con 0x7f5d6806a830 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.828+0000 7f5d5f7fe640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 0x7f5d540a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.828+0000 7f5d6659c640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 0x7f5d540a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.828+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f5d500f07f0 con 0x7f5d6806a830 2024-03-16T08:26:16.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.829+0000 7f5d6659c640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 0x7f5d540a4e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009b50 tx=0x7f5d58005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:16.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:16.833+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5d500bd680 con 0x7f5d6806a830 2024-03-16T08:26:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.056+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d681a7060 con 0x7f5d6806a830 2024-03-16T08:26:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.056+0000 7f5d5f7fe640 1 -- 172.21.15.49:0/4117044092 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f5d500bcf80 con 0x7f5d6806a830 2024-03-16T08:26:17.057 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:26:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:26:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 msgr2=0x7f5d540a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 0x7f5d540a4e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009b50 tx=0x7f5d58005f90 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 msgr2=0x7f5d6819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005940 tx=0x7f5d5002cbd0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 shutdown_connections 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5d540a2970 0x7f5d540a4e30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d681077e0 0x7f5d6819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d6806a830 0x7f5d6819a860 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 --2- 172.21.15.49:0/4117044092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d68069e90 0x7f5d6819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 >> 172.21.15.49:0/4117044092 conn(0x7f5d68077780 msgr2=0x7f5d6810c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 shutdown_connections 2024-03-16T08:26:17.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.058+0000 7f5d6cb62640 1 -- 172.21.15.49:0/4117044092 wait complete. 2024-03-16T08:26:17.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:26:17.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 -- 172.21.15.49:0/841203005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc100700 msgr2=0x7faddc100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 --2- 172.21.15.49:0/841203005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc100700 0x7faddc100b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009890 tx=0x7fadcc02bb70 comp rx=0 tx=0).stop 2024-03-16T08:26:17.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 -- 172.21.15.49:0/841203005 shutdown_connections 2024-03-16T08:26:17.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 --2- 172.21.15.49:0/841203005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc1010c0 0x7faddc10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 --2- 172.21.15.49:0/841203005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc100700 0x7faddc100b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 --2- 172.21.15.49:0/841203005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 -- 172.21.15.49:0/841203005 >> 172.21.15.49:0/841203005 conn(0x7faddc077840 msgr2=0x7faddc077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 -- 172.21.15.49:0/841203005 shutdown_connections 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.180+0000 7fade2ce8640 1 -- 172.21.15.49:0/841203005 wait complete. 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 Processor -- start 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 -- start start 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc100700 0x7faddc0fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc1010c0 0x7faddc0fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faddc110b20 con 0x7faddc106710 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1109a0 con 0x7faddc100700 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade2ce8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faddc110ca0 con 0x7faddc1010c0 2024-03-16T08:26:17.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58544/0 (socket says 172.21.15.49:58544) 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 -- 172.21.15.49:0/23902996 learned_addr learned my addr 172.21.15.49:0/23902996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade0a84640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc100700 0x7faddc0fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7faddbfff640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc1010c0 0x7faddc0fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 -- 172.21.15.49:0/23902996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc1010c0 msgr2=0x7faddc0fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc1010c0 0x7faddc0fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 -- 172.21.15.49:0/23902996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc100700 msgr2=0x7faddc0fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc100700 0x7faddc0fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.181+0000 7fade1285640 1 -- 172.21.15.49:0/23902996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a6610 con 0x7faddc106710 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fade1285640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fadd400eb10 tx=0x7fadd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fadd9ffb640 1 -- 172.21.15.49:0/23902996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004b20 con 0x7faddc106710 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a6900 con 0x7faddc106710 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fadd9ffb640 1 -- 172.21.15.49:0/23902996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4004cc0 con 0x7faddc106710 2024-03-16T08:26:17.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fadd9ffb640 1 -- 172.21.15.49:0/23902996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4010760 con 0x7faddc106710 2024-03-16T08:26:17.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.182+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a6e20 con 0x7faddc106710 2024-03-16T08:26:17.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fadd9ffb640 1 -- 172.21.15.49:0/23902996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fadd4005000 con 0x7faddc106710 2024-03-16T08:26:17.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 Processor -- start 2024-03-16T08:26:17.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 -- start start 2024-03-16T08:26:17.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd44bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd457120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fadd9ffb640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 0x7fadc80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ffd3af490 con 0x562ffd454d30 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ffd357c90 con 0x562ffd44c150 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade59aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ffd3ab530 con 0x562ffd44b830 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade0a84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7faddbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd457120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade1285640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 0x7fadc80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade0a84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51864/0 (socket says 172.21.15.49:51864) 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fade0a84640 1 -- 172.21.15.49:0/599095512 learned_addr learned my addr 172.21.15.49:0/599095512 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7faddbfff640 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd44bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.185+0000 7fadd9ffb640 1 -- 172.21.15.49:0/23902996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fadd40cf4b0 con 0x7faddc106710 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 -- 172.21.15.49:0/599095512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 msgr2=0x562ffd44bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd44bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 -- 172.21.15.49:0/599095512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 msgr2=0x562ffd457120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd457120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 -- 172.21.15.49:0/599095512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffd412500 con 0x562ffd44c150 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade1285640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 0x7fadc80a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fadd40049d0 tx=0x7fadd4007850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fade0a84640 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c60 tx=0x7fadc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840dd01dbd7f9406 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fadd2ffd640 1 -- 172.21.15.49:0/599095512 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4009d50 con 0x562ffd44c150 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fadd2ffd640 1 -- 172.21.15.49:0/599095512 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc4005500 con 0x562ffd44c150 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.186+0000 7fadd2ffd640 1 -- 172.21.15.49:0/599095512 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4019670 con 0x562ffd44c150 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- 172.21.15.49:0/599095512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 msgr2=0x562ffd4547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c60 tx=0x7fadc4011810 comp rx=0 tx=0).stop 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- 172.21.15.49:0/599095512 shutdown_connections 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd457120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44c150 0x562ffd4547f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 --2- 172.21.15.49:0/599095512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd44bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- 172.21.15.49:0/599095512 >> 172.21.15.49:0/599095512 conn(0x562ffd445f60 msgr2=0x562ffd446340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- 172.21.15.49:0/599095512 shutdown_connections 2024-03-16T08:26:17.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- 172.21.15.49:0/599095512 wait complete. 2024-03-16T08:26:17.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 -- start start 2024-03-16T08:26:17.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.187+0000 7fade59aa740 1 Processor -- start 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd451c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd452710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ffd457b20 con 0x562ffd44c150 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ffd36b230 con 0x562ffd454d30 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ffd457ca0 con 0x562ffd44b830 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade59aa740 10 client.0 _ll_register_callbacks cb 0x562ffd3a7010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58564/0 (socket says 172.21.15.49:58564) 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 -- 172.21.15.49:0/3944340043 learned_addr learned my addr 172.21.15.49:0/3944340043 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7faddbfff640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd452710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7faddbfff640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd451c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 -- 172.21.15.49:0/3944340043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 msgr2=0x562ffd451c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd451c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 -- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 msgr2=0x562ffd452710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd452710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 -- 172.21.15.49:0/3944340043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffd37a100 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fade0a84640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c30 tx=0x7fadc4005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b6cba8c50f685d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4028070 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.188+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fadc4005120 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc4005500 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fadc4033050 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 10 client.17739 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ffd0c33f0 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc4005500 con 0x562ffd44c150 2024-03-16T08:26:17.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fadc4022410 con 0x562ffd44c150 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 10 client.17739 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 20 client.17739 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd464ed0 0x562ffd4672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 4 client.17739 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffd3bc9d0 con 0x562ffd464ed0 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.189+0000 7faddbfff640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd464ed0 0x562ffd4672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7faddbfff640 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd464ed0 0x562ffd4672c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=437ebebdde3f671b server_cookie=a7bfd35aa69d20eb in_seq=0 out_seq=0 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fadbbfff640 10 client.17739 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fadbbfff640 1 -- 172.21.15.49:0/3944340043 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ffd3bc9d0 con 0x562ffd464ed0 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fadbbfff640 10 client.17739 handle_command_reply: tid=0 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 client.17739 shutdown 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 msgr2=0x562ffd4521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c30 tx=0x7fadc4005df0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 shutdown_connections 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd464ed0 0x562ffd4672c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd452710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44c150 0x562ffd4521d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fadbbfff640 0 client.17739 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.190+0000 7fade59aa740 1 --2- 172.21.15.49:0/3944340043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44b830 0x562ffd451c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.191+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 >> 172.21.15.49:0/3944340043 conn(0x562ffd445f60 msgr2=0x562ffd447cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.191+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 shutdown_connections 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.191+0000 7fade59aa740 1 -- 172.21.15.49:0/3944340043 wait complete. 2024-03-16T08:26:17.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.191+0000 7fade59aa740 20 client.17739 trim_cache size 0 max 16384 2024-03-16T08:26:17.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 Processor -- start 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 -- start start 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd4608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd46caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b1f60 con 0x562ffd44b830 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1af830 con 0x562ffd454d30 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade59aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ad6a0 con 0x562ffd44c150 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7faddbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7fade0a84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd4608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7faddbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58570/0 (socket says 172.21.15.49:58570) 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7faddbfff640 1 -- 172.21.15.49:0/3278296086 learned_addr learned my addr 172.21.15.49:0/3278296086 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.204+0000 7faddbfff640 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd46caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 -- 172.21.15.49:0/3278296086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 msgr2=0x562ffd4608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd4608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 -- 172.21.15.49:0/3278296086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 msgr2=0x562ffd46caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd46caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 -- 172.21.15.49:0/3278296086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffd18d7d0 con 0x562ffd44b830 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7fade0a84640 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd4608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7faddbfff640 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fadcc000c00 tx=0x7fadcc02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5eaf977b3a0b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7fadd27fc640 1 -- 172.21.15.49:0/3278296086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc039040 con 0x562ffd44b830 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7fadd27fc640 1 -- 172.21.15.49:0/3278296086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadcc057070 con 0x562ffd44b830 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.205+0000 7fadd27fc640 1 -- 172.21.15.49:0/3278296086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0346a0 con 0x562ffd44b830 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- 172.21.15.49:0/3278296086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 msgr2=0x562ffd4543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fadcc000c00 tx=0x7fadcc02bb70 comp rx=0 tx=0).stop 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- 172.21.15.49:0/3278296086 shutdown_connections 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd454d30 0x562ffd46caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd4608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 --2- 172.21.15.49:0/3278296086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd44b830 0x562ffd4543c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- 172.21.15.49:0/3278296086 >> 172.21.15.49:0/3278296086 conn(0x562ffd445f60 msgr2=0x562ffd447c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- 172.21.15.49:0/3278296086 shutdown_connections 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- 172.21.15.49:0/3278296086 wait complete. 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 -- start start 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 Processor -- start 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.206+0000 7fade59aa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44b830 0x562ffd45d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd458ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1adcf0 con 0x562ffd454d30 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1af160 con 0x562ffd44b830 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ae3c0 con 0x562ffd44c150 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade59aa740 10 client.0 _ll_register_callbacks cb 0x7fadc4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7faddbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd458ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:40504/0 (socket says 172.21.15.49:40504) 2024-03-16T08:26:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7faddbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd458ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58574/0 (socket says 172.21.15.49:58574) 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 -- 172.21.15.49:0/2101670364 learned_addr learned my addr 172.21.15.49:0/2101670364 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7faddbfff640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44b830 0x562ffd45d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 -- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44b830 msgr2=0x562ffd45d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44b830 0x562ffd45d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 -- 172.21.15.49:0/2101670364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 msgr2=0x562ffd458ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd458ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 -- 172.21.15.49:0/2101670364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffd459300 con 0x562ffd44c150 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fade0a84640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadc4022860 tx=0x7fadc4004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1dcc15e00fb2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.207+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:26:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.209+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4028040 con 0x562ffd44c150 2024-03-16T08:26:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.209+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fadc4022c50 con 0x562ffd44c150 2024-03-16T08:26:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.209+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadcc057070 con 0x562ffd44c150 2024-03-16T08:26:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.212+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fadc4004d70 con 0x562ffd44c150 2024-03-16T08:26:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.212+0000 7fade59aa740 10 client.27628 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ffd4cb180 con 0x562ffd44c150 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadcc057070 con 0x562ffd44c150 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fadc4024780 con 0x562ffd44c150 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 10 client.27628 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 10 client.27628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 20 client.27628 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd4cb840 0x562ffd4cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 4 client.27628 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ffd3b6d20 con 0x562ffd4cb840 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.213+0000 7faddbfff640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd4cb840 0x562ffd4cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fadbbfff640 10 client.27628 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7faddbfff640 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd4cb840 0x562ffd4cdc30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c824912690dc066a server_cookie=5aebbe65ea949f58 in_seq=0 out_seq=0 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fadbbfff640 1 -- 172.21.15.49:0/2101670364 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ffd3b6d20 con 0x562ffd4cb840 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fadbbfff640 10 client.27628 handle_command_reply: tid=0 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 client.27628 shutdown 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 msgr2=0x562ffd458590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadc4022860 tx=0x7fadc4004c00 comp rx=0 tx=0).stop 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 shutdown_connections 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562ffd4cb840 0x562ffd4cdc30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ffd454d30 0x562ffd458ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ffd44c150 0x562ffd458590 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 --2- 172.21.15.49:0/2101670364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffd44b830 0x562ffd45d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fadbbfff640 0 client.27628 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 >> 172.21.15.49:0/2101670364 conn(0x562ffd445f60 msgr2=0x562ffd451c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 shutdown_connections 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 1 -- 172.21.15.49:0/2101670364 wait complete. 2024-03-16T08:26:17.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.214+0000 7fade59aa740 20 client.27628 trim_cache size 0 max 16384 2024-03-16T08:26:17.217 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 msgr2=0x7fadc80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 0x7fadc80a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fadd40049d0 tx=0x7fadd4007850 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 msgr2=0x7faddc0fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fadd400eb10 tx=0x7fadd400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 shutdown_connections 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fadc80a29a0 0x7fadc80a4e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faddc106710 0x7faddc0fe100 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faddc1010c0 0x7faddc0fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 --2- 172.21.15.49:0/23902996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faddc100700 0x7faddc0fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 >> 172.21.15.49:0/23902996 conn(0x7faddc077840 msgr2=0x7faddc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 shutdown_connections 2024-03-16T08:26:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.215+0000 7fade2ce8640 1 -- 172.21.15.49:0/23902996 wait complete. 2024-03-16T08:26:17.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- 172.21.15.49:0/2437246100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364107760 msgr2=0x7f8364107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- 172.21.15.49:0/2437246100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364107760 0x7f8364107b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f835c009890 tx=0x7f835c02bb70 comp rx=0 tx=0).stop 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- 172.21.15.49:0/2437246100 shutdown_connections 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- 172.21.15.49:0/2437246100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8364069380 0x7f836410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- 172.21.15.49:0/2437246100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83640689c0 0x7f8364068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- 172.21.15.49:0/2437246100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364107760 0x7f8364107b40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- 172.21.15.49:0/2437246100 >> 172.21.15.49:0/2437246100 conn(0x7f83640777e0 msgr2=0x7f8364077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- 172.21.15.49:0/2437246100 shutdown_connections 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- 172.21.15.49:0/2437246100 wait complete. 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 Processor -- start 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- start start 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 0x7f83641006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364069380 0x7f8364100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111b30 con 0x7f8364107760 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83641119b0 con 0x7f83640689c0 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.336+0000 7f836b297640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111cb0 con 0x7f8364069380 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369834640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 0x7f83641006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369834640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58596/0 (socket says 172.21.15.49:58596) 2024-03-16T08:26:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369834640 1 -- 172.21.15.49:0/2845433756 learned_addr learned my addr 172.21.15.49:0/2845433756 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8368832640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364069380 0x7f8364100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 -- 172.21.15.49:0/2845433756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364069380 msgr2=0x7f8364100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364069380 0x7f8364100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 -- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 msgr2=0x7f8364101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 -- 172.21.15.49:0/2845433756 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83641a65d0 con 0x7f83640689c0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369834640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f8369033640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 0x7f83641006a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f835c02c180 tx=0x7f835c0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f83627fc640 1 -- 172.21.15.49:0/2845433756 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c041020 con 0x7f83640689c0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83641a6860 con 0x7f83640689c0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f83627fc640 1 -- 172.21.15.49:0/2845433756 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f835c02c9f0 con 0x7f83640689c0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f83627fc640 1 -- 172.21.15.49:0/2845433756 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c039040 con 0x7f83640689c0 2024-03-16T08:26:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.337+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641a6da0 con 0x7f83640689c0 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.339+0000 7f83627fc640 1 -- 172.21.15.49:0/2845433756 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f835c044020 con 0x7f83640689c0 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 Processor -- start 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 -- start start 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f414830 0x55a98f414c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f415150 0x55a98f41d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a98f378490 con 0x55a98f41dd30 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a98f320c90 con 0x55a98f415150 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f836de81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a98f374530 con 0x55a98f414830 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f83627fc640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 0x7f834c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f8368832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.340+0000 7f8369033640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f415150 0x55a98f41d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8369834640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f414830 0x55a98f414c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58598/0 (socket says 172.21.15.49:58598) 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 -- 172.21.15.49:0/2644275678 learned_addr learned my addr 172.21.15.49:0/2644275678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8369834640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 0x7f834c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f83627fc640 1 -- 172.21.15.49:0/2845433756 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f835c0f0520 con 0x7f83640689c0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 -- 172.21.15.49:0/2644275678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f414830 msgr2=0x55a98f414c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f414830 0x55a98f414c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 -- 172.21.15.49:0/2644275678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f415150 msgr2=0x55a98f41d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f415150 0x55a98f41d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 -- 172.21.15.49:0/2644275678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98f3db500 con 0x55a98f41dd30 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8368832640 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c80 tx=0x7f8358011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e118ac9ab3246b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f8369834640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 0x7f834c0a4e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8354009aa0 tx=0x7f8354009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f834b7fe640 1 -- 172.21.15.49:0/2644275678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358004220 con 0x55a98f41dd30 2024-03-16T08:26:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f834b7fe640 1 -- 172.21.15.49:0/2644275678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358009e30 con 0x55a98f41dd30 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.341+0000 7f834b7fe640 1 -- 172.21.15.49:0/2644275678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358019890 con 0x55a98f41dd30 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- 172.21.15.49:0/2644275678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 msgr2=0x55a98f420120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c80 tx=0x7f8358011810 comp rx=0 tx=0).stop 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- 172.21.15.49:0/2644275678 shutdown_connections 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f420120 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f415150 0x55a98f41d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 --2- 172.21.15.49:0/2644275678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f414830 0x55a98f414c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- 172.21.15.49:0/2644275678 >> 172.21.15.49:0/2644275678 conn(0x55a98f40ef60 msgr2=0x55a98f40f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- 172.21.15.49:0/2644275678 shutdown_connections 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- 172.21.15.49:0/2644275678 wait complete. 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 -- start start 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.342+0000 7f836de81740 1 Processor -- start 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f41b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f41dd30 0x55a98f41b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a98f420b20 con 0x55a98f414830 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a98f334230 con 0x55a98f41dd30 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a98f420ca0 con 0x55a98f415150 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f836de81740 10 client.0 _ll_register_callbacks cb 0x55a98f370010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58608/0 (socket says 172.21.15.49:58608) 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 -- 172.21.15.49:0/673451092 learned_addr learned my addr 172.21.15.49:0/673451092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8368832640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f41b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369834640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f41dd30 0x55a98f41b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 -- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 msgr2=0x55a98f41b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f41b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 -- 172.21.15.49:0/673451092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f41dd30 msgr2=0x55a98f41b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f41dd30 0x55a98f41b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 -- 172.21.15.49:0/673451092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98f343100 con 0x55a98f414830 2024-03-16T08:26:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8369033640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f835c0b90b0 tx=0x7f835c08ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d82c6ebb3aafa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.343+0000 7f8348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c032640 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f835c032d90 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358009e30 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f835c0bd350 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 10 client.17757 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a98f08c3f0 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358009e30 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f835c03c030 con 0x55a98f414830 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 10 client.17757 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 10 client.17757 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 20 client.17757 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f42e000 0x55a98f430430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 4 client.17757 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:17.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98f3859d0 con 0x55a98f42e000 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.344+0000 7f8368832640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f42e000 0x55a98f430430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f8348ff9640 10 client.17757 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f8368832640 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f42e000 0x55a98f430430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7e9a5493922342e server_cookie=269081941814dba3 in_seq=0 out_seq=0 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f8348ff9640 1 -- 172.21.15.49:0/673451092 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a98f3859d0 con 0x55a98f42e000 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f8348ff9640 10 client.17757 handle_command_reply: tid=0 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f836de81740 1 client.17757 shutdown 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 msgr2=0x55a98f41ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.345+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f835c0b90b0 tx=0x7f835c08ba60 comp rx=0 tx=0).stop 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 shutdown_connections 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f42e000 0x55a98f430430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f41dd30 0x55a98f41b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f41b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 --2- 172.21.15.49:0/673451092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f414830 0x55a98f41ace0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f8348ff9640 0 client.17757 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:17.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 >> 172.21.15.49:0/673451092 conn(0x55a98f40ef60 msgr2=0x55a98f410cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 shutdown_connections 2024-03-16T08:26:17.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 1 -- 172.21.15.49:0/673451092 wait complete. 2024-03-16T08:26:17.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.346+0000 7f836de81740 20 client.17757 trim_cache size 0 max 16384 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 Processor -- start 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 -- start start 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f435cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b1f20 con 0x55a98f41dd30 2024-03-16T08:26:17.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83641af7f0 con 0x55a98f414830 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f836de81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ad660 con 0x55a98f415150 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f8369834640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f8369834640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58622/0 (socket says 172.21.15.49:58622) 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f8369834640 1 -- 172.21.15.49:0/1283447133 learned_addr learned my addr 172.21.15.49:0/1283447133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f8369033640 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.359+0000 7f8368832640 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f435cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 -- 172.21.15.49:0/1283447133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 msgr2=0x55a98f435cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f435cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 -- 172.21.15.49:0/1283447133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 msgr2=0x55a98f4298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4298c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 -- 172.21.15.49:0/1283447133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98f1567d0 con 0x55a98f41dd30 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f8369834640 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8354014010 tx=0x7f8354018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863539cc87f8afd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f834affd640 1 -- 172.21.15.49:0/1283447133 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354010e30 con 0x55a98f41dd30 2024-03-16T08:26:17.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f834affd640 1 -- 172.21.15.49:0/1283447133 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8354005020 con 0x55a98f41dd30 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f834affd640 1 -- 172.21.15.49:0/1283447133 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835402b430 con 0x55a98f41dd30 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f836de81740 1 -- 172.21.15.49:0/1283447133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 msgr2=0x55a98f40de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.360+0000 7f836de81740 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8354014010 tx=0x7f8354018000 comp rx=0 tx=0).stop 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- 172.21.15.49:0/1283447133 shutdown_connections 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f41dd30 0x55a98f40de60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f415150 0x55a98f435cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- 172.21.15.49:0/1283447133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- 172.21.15.49:0/1283447133 >> 172.21.15.49:0/1283447133 conn(0x55a98f40ef60 msgr2=0x55a98f40d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- 172.21.15.49:0/1283447133 shutdown_connections 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- 172.21.15.49:0/1283447133 wait complete. 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- start start 2024-03-16T08:26:17.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 Processor -- start 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f41dd30 0x55a98f417b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83641adcb0 con 0x55a98f415150 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83641af120 con 0x55a98f414830 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ae380 con 0x55a98f41dd30 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f836de81740 10 client.0 _ll_register_callbacks cb 0x7f835c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f8368832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f8368832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58624/0 (socket says 172.21.15.49:58624) 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.361+0000 7f8368832640 1 -- 172.21.15.49:0/1969510368 learned_addr learned my addr 172.21.15.49:0/1969510368 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8369033640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8369834640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f41dd30 0x55a98f417b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 -- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f41dd30 msgr2=0x55a98f417b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f41dd30 0x55a98f417b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 -- 172.21.15.49:0/1969510368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 msgr2=0x55a98f4261c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4261c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 -- 172.21.15.49:0/1969510368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98f4183a0 con 0x55a98f415150 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8368832640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f83580092a0 tx=0x7f8358003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e4c0d7eeceace8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358005d10 con 0x55a98f415150 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f83580041d0 con 0x55a98f415150 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8354005020 con 0x55a98f415150 2024-03-16T08:26:17.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f83580043d0 con 0x55a98f415150 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f836de81740 10 client.17763 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a98f494310 con 0x55a98f415150 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.362+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8354005020 con 0x55a98f415150 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8358004660 con 0x55a98f415150 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 10 client.17763 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 10 client.17763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 20 client.17763 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f494930 0x55a98f496d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 4 client.17763 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a98f37fd20 con 0x55a98f494930 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8369033640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f494930 0x55a98f496d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8369033640 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f494930 0x55a98f496d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=999bec39b0f65fe0 server_cookie=e634ab227ecfddbe in_seq=0 out_seq=0 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8348ff9640 10 client.17763 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8348ff9640 1 -- 172.21.15.49:0/1969510368 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a98f37fd20 con 0x55a98f494930 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f8348ff9640 10 client.17763 handle_command_reply: tid=0 2024-03-16T08:26:17.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 1 client.17763 shutdown 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 msgr2=0x55a98f417630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.363+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f83580092a0 tx=0x7f8358003f50 comp rx=0 tx=0).stop 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 shutdown_connections 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55a98f494930 0x55a98f496d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a98f41dd30 0x55a98f417b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f8348ff9640 0 client.17763 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a98f415150 0x55a98f417630 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 --2- 172.21.15.49:0/1969510368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a98f414830 0x55a98f4261c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 >> 172.21.15.49:0/1969510368 conn(0x55a98f40ef60 msgr2=0x55a98f41b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 shutdown_connections 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 1 -- 172.21.15.49:0/1969510368 wait complete. 2024-03-16T08:26:17.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836de81740 20 client.17763 trim_cache size 0 max 16384 2024-03-16T08:26:17.371 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:17.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 msgr2=0x7f834c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 0x7f834c0a4e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8354009aa0 tx=0x7f8354009500 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 msgr2=0x7f83641006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 0x7f83641006a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f835c02c180 tx=0x7f835c0045b0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 shutdown_connections 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f834c0a2970 0x7f834c0a4e30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8364107760 0x7f8364101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8364069380 0x7f8364100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 --2- 172.21.15.49:0/2845433756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83640689c0 0x7f83641006a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 >> 172.21.15.49:0/2845433756 conn(0x7f83640777e0 msgr2=0x7f83640fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 shutdown_connections 2024-03-16T08:26:17.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.364+0000 7f836b297640 1 -- 172.21.15.49:0/2845433756 wait complete. 2024-03-16T08:26:17.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:17 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4117044092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:17.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:26:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:17 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4117044092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.484+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/4130343331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4101780 msgr2=0x7fe8d4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.484+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/4130343331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4101780 0x7fe8d4101c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009930 tx=0x7fe8bc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/4130343331 shutdown_connections 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/4130343331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/4130343331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4101780 0x7fe8d4101c00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/4130343331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4107790 0x7fe8d4107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/4130343331 >> 172.21.15.49:0/4130343331 conn(0x7fe8d40fd210 msgr2=0x7fe8d40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/4130343331 shutdown_connections 2024-03-16T08:26:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/4130343331 wait complete. 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 Processor -- start 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.485+0000 7fe8d8d5b640 1 -- start start 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4107790 0x7fe8d419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111ba0 con 0x7fe8d4101780 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111a20 con 0x7fe8d4102140 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111d20 con 0x7fe8d4107790 2024-03-16T08:26:17.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58648/0 (socket says 172.21.15.49:58648) 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 -- 172.21.15.49:0/2902365369 learned_addr learned my addr 172.21.15.49:0/2902365369 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4107790 0x7fe8d419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 -- 172.21.15.49:0/2902365369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4107790 msgr2=0x7fe8d419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4107790 0x7fe8d419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 -- 172.21.15.49:0/2902365369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 msgr2=0x7fe8d4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 -- 172.21.15.49:0/2902365369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d419d0d0 con 0x7fe8d4101780 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d259c640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00ea60 tx=0x7fe8cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8cb7fe640 1 -- 172.21.15.49:0/2902365369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00cde0 con 0x7fe8d4101780 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d419d3c0 con 0x7fe8d4101780 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8cb7fe640 1 -- 172.21.15.49:0/2902365369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8cc0035f0 con 0x7fe8d4101780 2024-03-16T08:26:17.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8cb7fe640 1 -- 172.21.15.49:0/2902365369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc010670 con 0x7fe8d4101780 2024-03-16T08:26:17.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.486+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a4c50 con 0x7fe8d4101780 2024-03-16T08:26:17.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8cb7fe640 1 -- 172.21.15.49:0/2902365369 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fe8cc0037a0 con 0x7fe8d4101780 2024-03-16T08:26:17.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 Processor -- start 2024-03-16T08:26:17.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 -- start start 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 0x555ec68cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555ec6832490 con 0x555ec68cf150 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ec67dac90 con 0x555ec68d7d30 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8dba22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555ec682e530 con 0x555ec68ce830 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8cb7fe640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 0x7fe8c00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 0x555ec68cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.489+0000 7fe8d1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51942/0 (socket says 172.21.15.49:51942) 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/1827463544 learned_addr learned my addr 172.21.15.49:0/1827463544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 0x555ec68cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:40596/0 (socket says 172.21.15.49:40596) 2024-03-16T08:26:17.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 0x7fe8c00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8cb7fe640 1 -- 172.21.15.49:0/2902365369 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fe8cc0cf1b0 con 0x7fe8d4101780 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/1827463544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 msgr2=0x555ec68cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 0x555ec68cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/1827463544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 msgr2=0x555ec68d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/1827463544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec6895500 con 0x555ec68d7d30 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 0x7fe8c00a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009270 tx=0x7fe8c4009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b85e054367fdce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8c8ff9640 1 -- 172.21.15.49:0/1827463544 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc02ca80 con 0x555ec68d7d30 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8c8ff9640 1 -- 172.21.15.49:0/1827463544 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc02cc20 con 0x555ec68d7d30 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.490+0000 7fe8c8ff9640 1 -- 172.21.15.49:0/1827463544 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0048e0 con 0x555ec68d7d30 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- 172.21.15.49:0/1827463544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 msgr2=0x555ec68da120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc005df0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- 172.21.15.49:0/1827463544 shutdown_connections 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68d7d30 0x555ec68da120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1827463544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68ce830 0x555ec68cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- 172.21.15.49:0/1827463544 >> 172.21.15.49:0/1827463544 conn(0x555ec68c8f60 msgr2=0x555ec68c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- 172.21.15.49:0/1827463544 shutdown_connections 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- 172.21.15.49:0/1827463544 wait complete. 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 -- start start 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.491+0000 7fe8dba22740 1 Processor -- start 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 0x555ec68d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555ec68daac0 con 0x555ec68d7d30 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ec67ee230 con 0x555ec68ce830 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555ec68dac40 con 0x555ec68cf150 2024-03-16T08:26:17.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8dba22740 10 client.0 _ll_register_callbacks cb 0x555ec682a010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58666/0 (socket says 172.21.15.49:58666) 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 -- 172.21.15.49:0/1496755988 learned_addr learned my addr 172.21.15.49:0/1496755988 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 0x555ec68d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d259c640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 -- 172.21.15.49:0/1496755988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 msgr2=0x555ec68d52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 0x555ec68d52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 -- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 msgr2=0x555ec68d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 -- 172.21.15.49:0/1496755988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec67fd100 con 0x555ec68d7d30 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 0x555ec68d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d259c640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4012fb0 tx=0x7fe8c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f467e7cc51aa7373 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.492+0000 7fe8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4003e00 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fe8c40047d0 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8bc02cc20 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe8c4004b20 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 10 client.17775 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555ec65463f0 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc02cc20 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe8c4030020 con 0x555ec68d7d30 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 10 client.17775 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 20 client.17775 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec68e8130 0x555ec68ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 4 client.17775 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:17.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ec683f9d0 con 0x555ec68e8130 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.493+0000 7fe8d259c640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec68e8130 0x555ec68ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8b9ffb640 10 client.17775 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8d259c640 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec68e8130 0x555ec68ea570 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c932893f84da89b9 server_cookie=a2e858a87c8a8e80 in_seq=0 out_seq=0 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/1496755988 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ec683f9d0 con 0x555ec68e8130 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8b9ffb640 10 client.17775 handle_command_reply: tid=0 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8dba22740 1 client.17775 shutdown 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 msgr2=0x555ec68d5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.494+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4012fb0 tx=0x7fe8c4004ab0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 shutdown_connections 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec68e8130 0x555ec68ea570 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68d7d30 0x555ec68d5810 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68cf150 0x555ec68d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8b9ffb640 0 client.17775 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:17.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 --2- 172.21.15.49:0/1496755988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 >> 172.21.15.49:0/1496755988 conn(0x555ec68c8f60 msgr2=0x555ec68cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 shutdown_connections 2024-03-16T08:26:17.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 1 -- 172.21.15.49:0/1496755988 wait complete. 2024-03-16T08:26:17.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.495+0000 7fe8dba22740 20 client.17775 trim_cache size 0 max 16384 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 Processor -- start 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 -- start start 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68e3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41afdf0 con 0x555ec68cf150 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ad6c0 con 0x555ec68ce830 2024-03-16T08:26:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8dba22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ab530 con 0x555ec68d7d30 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58668/0 (socket says 172.21.15.49:58668) 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:51958/0 (socket says 172.21.15.49:51958) 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/4255476744 learned_addr learned my addr 172.21.15.49:0/4255476744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.508+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/4255476744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 msgr2=0x555ec68e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/4255476744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 msgr2=0x555ec68d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 -- 172.21.15.49:0/4255476744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec66107d0 con 0x555ec68cf150 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d259c640 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0054d0 tx=0x7fe8bc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289e9c8ae81f1c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8bbfff640 1 -- 172.21.15.49:0/4255476744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc02ca80 con 0x555ec68cf150 2024-03-16T08:26:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8bbfff640 1 -- 172.21.15.49:0/4255476744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc005630 con 0x555ec68cf150 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8bbfff640 1 -- 172.21.15.49:0/4255476744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc003e00 con 0x555ec68cf150 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8dba22740 1 -- 172.21.15.49:0/4255476744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 msgr2=0x555ec68e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.509+0000 7fe8dba22740 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0054d0 tx=0x7fe8bc005fd0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- 172.21.15.49:0/4255476744 shutdown_connections 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68cf150 0x555ec68e0270 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- 172.21.15.49:0/4255476744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68ce830 0x555ec68d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- 172.21.15.49:0/4255476744 >> 172.21.15.49:0/4255476744 conn(0x555ec68c8f60 msgr2=0x555ec68cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- 172.21.15.49:0/4255476744 shutdown_connections 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- 172.21.15.49:0/4255476744 wait complete. 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- start start 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 Processor -- start 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68cf150 0x555ec68efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41abb80 con 0x555ec68ce830 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41acff0 con 0x555ec68cf150 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ac250 con 0x555ec68d7d30 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8dba22740 10 client.0 _ll_register_callbacks cb 0x7fe8c4005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58674/0 (socket says 172.21.15.49:58674) 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8d259c640 1 -- 172.21.15.49:0/426808848 learned_addr learned my addr 172.21.15.49:0/426808848 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68cf150 0x555ec68efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.510+0000 7fe8d2d9d640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 -- 172.21.15.49:0/426808848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 msgr2=0x555ec68f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 -- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68cf150 msgr2=0x555ec68efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68cf150 0x555ec68efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 -- 172.21.15.49:0/426808848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec69514a0 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8d259c640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc09b7c0 tx=0x7fe8cc06d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0e09c0350dc99b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc06a070 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fe8cc003b30 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ec6839d20 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe8cc0860b0 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8dba22740 10 client.17781 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555ec69527e0 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555ec6839d20 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe8cc0a3b40 con 0x555ec68ce830 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8dba22740 10 client.17781 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.511+0000 7fe8dba22740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 20 client.17781 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec6952f90 0x555ec6955380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 4 client.17781 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ec67b76a0 con 0x555ec6952f90 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec6952f90 0x555ec6955380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8d1d9b640 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec6952f90 0x555ec6955380 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb987825dfcb82e server_cookie=f32f8cada0bf3e3c in_seq=0 out_seq=0 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8b9ffb640 10 client.17781 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8b9ffb640 1 -- 172.21.15.49:0/426808848 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ec67b76a0 con 0x555ec6952f90 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8b9ffb640 10 client.17781 handle_command_reply: tid=0 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 client.17781 shutdown 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 msgr2=0x555ec68ef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc09b7c0 tx=0x7fe8cc06d460 comp rx=0 tx=0).stop 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 shutdown_connections 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x555ec6952f90 0x555ec6955380 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ec68d7d30 0x555ec68f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8b9ffb640 0 client.17781 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ec68cf150 0x555ec68efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 --2- 172.21.15.49:0/426808848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555ec68ce830 0x555ec68ef740 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 >> 172.21.15.49:0/426808848 conn(0x555ec68c8f60 msgr2=0x555ec68d4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.512+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 shutdown_connections 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8dba22740 1 -- 172.21.15.49:0/426808848 wait complete. 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8dba22740 20 client.17781 trim_cache size 0 max 16384 2024-03-16T08:26:17.520 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 msgr2=0x7fe8c00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 0x7fe8c00a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009270 tx=0x7fe8c4009500 comp rx=0 tx=0).stop 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 msgr2=0x7fe8d41980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00ea60 tx=0x7fe8cc00ef20 comp rx=0 tx=0).stop 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 shutdown_connections 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe8c00a2970 0x7fe8c00a4e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d4107790 0x7fe8d419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d4102140 0x7fe8d4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 --2- 172.21.15.49:0/2902365369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8d4101780 0x7fe8d41980e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:17.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 >> 172.21.15.49:0/2902365369 conn(0x7fe8d40fd210 msgr2=0x7fe8d40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:17.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 shutdown_connections 2024-03-16T08:26:17.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:17.513+0000 7fe8d8d5b640 1 -- 172.21.15.49:0/2902365369 wait complete. 2024-03-16T08:26:17.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:17 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4117044092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:18.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:18 smithi154 ceph-mon[36471]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-03-16T08:26:18.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:26:18.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:26:18.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:26:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:18 smithi049 ceph-mon[31942]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-03-16T08:26:18.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:18 smithi129 ceph-mon[36561]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-03-16T08:26:18.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 -- 172.21.15.49:0/3151168616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 msgr2=0x7f3ae4100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 --2- 172.21.15.49:0/3151168616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae4100ad0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009890 tx=0x7f3ad402bb70 comp rx=0 tx=0).stop 2024-03-16T08:26:18.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 -- 172.21.15.49:0/3151168616 shutdown_connections 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 --2- 172.21.15.49:0/3151168616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 --2- 172.21.15.49:0/3151168616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae4100ad0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 --2- 172.21.15.49:0/3151168616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 -- 172.21.15.49:0/3151168616 >> 172.21.15.49:0/3151168616 conn(0x7f3ae40777e0 msgr2=0x7f3ae4077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 -- 172.21.15.49:0/3151168616 shutdown_connections 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.651+0000 7f3aec24b640 1 -- 172.21.15.49:0/3151168616 wait complete. 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 Processor -- start 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 -- start start 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4110a30 con 0x7f3ae4106660 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41108b0 con 0x7f3ae4101010 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aec24b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4110bb0 con 0x7f3ae4100650 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58698/0 (socket says 172.21.15.49:58698) 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 -- 172.21.15.49:0/2301884279 learned_addr learned my addr 172.21.15.49:0/2301884279 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 -- 172.21.15.49:0/2301884279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 msgr2=0x7f3ae41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 -- 172.21.15.49:0/2301884279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 msgr2=0x7f3ae4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3aea7e8640 1 -- 172.21.15.49:0/2301884279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae419d0d0 con 0x7f3ae4106660 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.652+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3aea7e8640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00eb10 tx=0x7f3adc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3adaffd640 1 -- 172.21.15.49:0/2301884279 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00ce20 con 0x7f3ae4106660 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae419d3c0 con 0x7f3ae4106660 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3adaffd640 1 -- 172.21.15.49:0/2301884279 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3adc005020 con 0x7f3ae4106660 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3adaffd640 1 -- 172.21.15.49:0/2301884279 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc010690 con 0x7f3ae4106660 2024-03-16T08:26:18.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.653+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae41a4c50 con 0x7f3ae4106660 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.655+0000 7f3adaffd640 1 -- 172.21.15.49:0/2301884279 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f3adc005540 con 0x7f3ae4106660 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.655+0000 7f3aeee34740 1 Processor -- start 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.655+0000 7f3aeee34740 1 -- start start 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56519a8270d0 con 0x56519a8c4710 2024-03-16T08:26:18.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56519a7cfc70 con 0x56519a8d11b0 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aeee34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56519a823170 con 0x56519a8c3a30 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3adaffd640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 0x7f3ad00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58710/0 (socket says 172.21.15.49:58710) 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae97e6640 1 -- 172.21.15.49:0/4242436089 learned_addr learned my addr 172.21.15.49:0/4242436089 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3aea7e8640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 0x7f3ad00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/4242436089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 msgr2=0x56519a8d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3adaffd640 1 -- 172.21.15.49:0/2301884279 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f3adc0cf3e0 con 0x7f3ae4106660 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/4242436089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 msgr2=0x56519a8c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.656+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/4242436089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519a6037e0 con 0x56519a8c3a30 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8c3eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009f60 tx=0x7f3ae0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ab1b86e68f77bc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aea7e8640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 0x7f3ad00a4e30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0ccc00 tx=0x7f3adc090ef0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3acffff640 1 -- 172.21.15.49:0/4242436089 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004230 con 0x56519a8c3a30 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3acffff640 1 -- 172.21.15.49:0/4242436089 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae00043d0 con 0x56519a8c3a30 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3acffff640 1 -- 172.21.15.49:0/4242436089 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000fa20 con 0x56519a8c3a30 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 -- 172.21.15.49:0/4242436089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 msgr2=0x56519a8c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8c3eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009f60 tx=0x7f3ae0009f90 comp rx=0 tx=0).stop 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 -- 172.21.15.49:0/4242436089 shutdown_connections 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8c3eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.657+0000 7f3aeee34740 1 --2- 172.21.15.49:0/4242436089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 -- 172.21.15.49:0/4242436089 >> 172.21.15.49:0/4242436089 conn(0x56519a88ac50 msgr2=0x56519a8be480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 -- 172.21.15.49:0/4242436089 shutdown_connections 2024-03-16T08:26:18.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 -- 172.21.15.49:0/4242436089 wait complete. 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 -- start start 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 Processor -- start 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.658+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 0x56519a8ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56519a8d4020 con 0x56519a8c4710 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56519a7e30d0 con 0x56519a8c3a30 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56519a8d41a0 con 0x56519a8d11b0 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58712/0 (socket says 172.21.15.49:58712) 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2802137614 learned_addr learned my addr 172.21.15.49:0/2802137614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 0x56519a8ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 10 client.0 _ll_register_callbacks cb 0x56519a7ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2802137614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 msgr2=0x56519a8ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 0x56519a8ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 msgr2=0x56519a8ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2802137614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519a663f70 con 0x56519a8c4710 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 0x56519a8ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3ae001b040 tx=0x7f3ae00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2ae72f8cb67f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0029020 con 0x56519a8c4710 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f3ae00037f0 con 0x56519a8c4710 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.659+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae00043d0 con 0x56519a8c4710 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0034050 con 0x56519a8c4710 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 10 client.17793 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:18.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56519a4fb170 con 0x56519a8c4710 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae00043d0 con 0x56519a8c4710 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3ae0003a80 con 0x56519a8c4710 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 10 client.17793 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 20 client.17793 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a8e11d0 0x56519a8e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 4 client.17793 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56519a834610 con 0x56519a8e11d0 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.660+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a8e11d0 0x56519a8e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3acd7fa640 10 client.17793 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a8e11d0 0x56519a8e35c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6c0390290ec5af server_cookie=818c279daacb5c74 in_seq=0 out_seq=0 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3acd7fa640 1 -- 172.21.15.49:0/2802137614 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56519a834610 con 0x56519a8e11d0 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3acd7fa640 10 client.17793 handle_command_reply: tid=0 2024-03-16T08:26:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3aeee34740 1 client.17793 shutdown 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 msgr2=0x56519a8ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.661+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3ae001b040 tx=0x7f3ae00052f0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 shutdown_connections 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a8e11d0 0x56519a8e35c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8d11b0 0x56519a8ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8ce630 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2802137614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3acd7fa640 0 client.17793 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 >> 172.21.15.49:0/2802137614 conn(0x56519a88ac50 msgr2=0x56519a8c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 shutdown_connections 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 1 -- 172.21.15.49:0/2802137614 wait complete. 2024-03-16T08:26:18.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.662+0000 7f3aeee34740 20 client.17793 trim_cache size 0 max 16384 2024-03-16T08:26:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 Processor -- start 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 -- start start 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41adbd0 con 0x56519a8c4710 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41ab4a0 con 0x56519a8d11b0 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3aeee34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41a9310 con 0x56519a8c3a30 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58716/0 (socket says 172.21.15.49:58716) 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:40676/0 (socket says 172.21.15.49:40676) 2024-03-16T08:26:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2852799851 learned_addr learned my addr 172.21.15.49:0/2852799851 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2852799851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 msgr2=0x56519a8d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2852799851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 msgr2=0x56519a8e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8e8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 -- 172.21.15.49:0/2852799851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519a8c2a60 con 0x56519a8c4710 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.680+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451265066104932e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3acf7fe640 1 -- 172.21.15.49:0/2852799851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0029040 con 0x56519a8c4710 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3acf7fe640 1 -- 172.21.15.49:0/2852799851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0009b90 con 0x56519a8c4710 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3acf7fe640 1 -- 172.21.15.49:0/2852799851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000fbd0 con 0x56519a8c4710 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 -- 172.21.15.49:0/2852799851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 msgr2=0x56519a8dce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0011d60 comp rx=0 tx=0).stop 2024-03-16T08:26:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 -- 172.21.15.49:0/2852799851 shutdown_connections 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8d11b0 0x56519a8e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8c4710 0x56519a8dce30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 --2- 172.21.15.49:0/2852799851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c3a30 0x56519a8d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 -- 172.21.15.49:0/2852799851 >> 172.21.15.49:0/2852799851 conn(0x56519a88ac50 msgr2=0x56519a8c00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 -- 172.21.15.49:0/2852799851 shutdown_connections 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.681+0000 7f3aeee34740 1 -- 172.21.15.49:0/2852799851 wait complete. 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 -- start start 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 Processor -- start 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c4710 0x56519a8cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41a9960 con 0x56519a8d11b0 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41aadd0 con 0x56519a8c3a30 2024-03-16T08:26:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41aa030 con 0x56519a8c4710 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3aeee34740 10 client.0 _ll_register_callbacks cb 0x7f3ae0011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3ae97e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58730/0 (socket says 172.21.15.49:58730) 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3ae97e6640 1 -- 172.21.15.49:0/3052366611 learned_addr learned my addr 172.21.15.49:0/3052366611 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.682+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c4710 0x56519a8cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 -- 172.21.15.49:0/3052366611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c4710 msgr2=0x56519a8cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c4710 0x56519a8cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 -- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 msgr2=0x56519a8cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 -- 172.21.15.49:0/3052366611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519a946910 con 0x56519a8d11b0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3ae97e6640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4005940 tx=0x7f3ad40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532c62334116194d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4039040 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f3ad4050ba0 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae0009b90 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ad4050da0 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3aeee34740 10 client.17799 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56519a947cb0 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.683+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0009b90 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3ad403f040 con 0x56519a8d11b0 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 10 client.17799 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 20 client.17799 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a9483b0 0x56519a94a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 4 client.17799 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:26:18.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56519a82e960 con 0x56519a9483b0 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a9483b0 0x56519a94a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3acd7fa640 10 client.17799 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.684+0000 7f3ae9fe7640 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a9483b0 0x56519a94a7d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db7774299113982 server_cookie=5855fd2e896b5c26 in_seq=0 out_seq=0 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3acd7fa640 1 -- 172.21.15.49:0/3052366611 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56519a82e960 con 0x56519a9483b0 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3acd7fa640 10 client.17799 handle_command_reply: tid=0 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 client.17799 shutdown 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 msgr2=0x56519a946380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4005940 tx=0x7f3ad40058d0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 shutdown_connections 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x56519a9483b0 0x56519a94a7d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56519a8d11b0 0x56519a946380 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56519a8c4710 0x56519a8cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 --2- 172.21.15.49:0/3052366611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56519a8c3a30 0x56519a8cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 >> 172.21.15.49:0/3052366611 conn(0x56519a88ac50 msgr2=0x56519a8bcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.685+0000 7f3acd7fa640 0 client.17799 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 shutdown_connections 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aeee34740 1 -- 172.21.15.49:0/3052366611 wait complete. 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aeee34740 20 client.17799 trim_cache size 0 max 16384 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "88f70423-25a1-4e58-ac78-83a6d7ffea79", 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:26:18.693 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 msgr2=0x7f3ad00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 0x7f3ad00a4e30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0ccc00 tx=0x7f3adc090ef0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 msgr2=0x7f3ae419c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00eb10 tx=0x7f3adc00ef10 comp rx=0 tx=0).stop 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 shutdown_connections 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f3ad00a2970 0x7f3ad00a4e30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ae4106660 0x7f3ae419c9a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae4101010 0x7f3ae4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 --2- 172.21.15.49:0/2301884279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ae4100650 0x7f3ae41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 >> 172.21.15.49:0/2301884279 conn(0x7f3ae40777e0 msgr2=0x7f3ae4068ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 shutdown_connections 2024-03-16T08:26:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.686+0000 7f3aec24b640 1 -- 172.21.15.49:0/2301884279 wait complete. 2024-03-16T08:26:18.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:26:18.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:26:18.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 -- 172.21.15.49:0/3031050504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 msgr2=0x7f5804107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 --2- 172.21.15.49:0/3031050504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f5804107b70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5800009930 tx=0x7f580002bc30 comp rx=0 tx=0).stop 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 -- 172.21.15.49:0/3031050504 shutdown_connections 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 --2- 172.21.15.49:0/3031050504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 0x7f580410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 --2- 172.21.15.49:0/3031050504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 0x7f5804101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 --2- 172.21.15.49:0/3031050504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f5804107b70 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 -- 172.21.15.49:0/3031050504 >> 172.21.15.49:0/3031050504 conn(0x7f58040fd210 msgr2=0x7f58040ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 -- 172.21.15.49:0/3031050504 shutdown_connections 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.850+0000 7f580b353640 1 -- 172.21.15.49:0/3031050504 wait complete. 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 Processor -- start 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 -- start start 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 0x7f580419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 0x7f580419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5804111ba0 con 0x7f5804107790 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5804111a20 con 0x7f5804102140 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f580b353640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5804111d20 con 0x7f5804101780 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58752/0 (socket says 172.21.15.49:58752) 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58088ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 0x7f580419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 -- 172.21.15.49:0/1330355972 learned_addr learned my addr 172.21.15.49:0/1330355972 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58090ef640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 0x7f580419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 -- 172.21.15.49:0/1330355972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 msgr2=0x7f580419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 0x7f580419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 -- 172.21.15.49:0/1330355972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 msgr2=0x7f580419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 0x7f580419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.851+0000 7f58098f0640 1 -- 172.21.15.49:0/1330355972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580419f2b0 con 0x7f5804107790 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f58098f0640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00eb10 tx=0x7f57fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f57fa7fc640 1 -- 172.21.15.49:0/1330355972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc004b20 con 0x7f5804107790 2024-03-16T08:26:18.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f57fa7fc640 1 -- 172.21.15.49:0/1330355972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc004cc0 con 0x7f5804107790 2024-03-16T08:26:18.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580419f5a0 con 0x7f5804107790 2024-03-16T08:26:18.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f57fa7fc640 1 -- 172.21.15.49:0/1330355972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc010760 con 0x7f5804107790 2024-03-16T08:26:18.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.852+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58041a6e30 con 0x7f5804107790 2024-03-16T08:26:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.854+0000 7f57fa7fc640 1 -- 172.21.15.49:0/1330355972 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f57fc005020 con 0x7f5804107790 2024-03-16T08:26:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.854+0000 7f580e018740 1 Processor -- start 2024-03-16T08:26:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.854+0000 7f580e018740 1 -- start start 2024-03-16T08:26:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f06110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531f06650 0x559531f08a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559531e61500 con 0x559531efd150 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559531e09be0 con 0x559531f06650 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f580e018740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559531e5d5a0 con 0x559531efda70 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58768/0 (socket says 172.21.15.49:58768) 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 -- 172.21.15.49:0/4013731669 learned_addr learned my addr 172.21.15.49:0/4013731669 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58090ef640 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f06110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531f06650 0x559531f08a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f57fa7fc640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 0x7f57f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 -- 172.21.15.49:0/4013731669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 msgr2=0x559531f06110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f06110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 -- 172.21.15.49:0/4013731669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531f06650 msgr2=0x559531f08a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531f06650 0x559531f08a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 -- 172.21.15.49:0/4013731669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559531e1d4b0 con 0x559531efd150 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58098f0640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 0x7f57f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f57fa7fc640 1 -- 172.21.15.49:0/1330355972 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f57fc0cf350 con 0x7f5804107790 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.855+0000 7f58088ee640 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013620 tx=0x7f57f4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbe6834f45fa99e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f57ef7fe640 1 -- 172.21.15.49:0/4013731669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4004050 con 0x559531efd150 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f57ef7fe640 1 -- 172.21.15.49:0/4013731669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f40041f0 con 0x559531efd150 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f57ef7fe640 1 -- 172.21.15.49:0/4013731669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f400f7e0 con 0x559531efd150 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 -- 172.21.15.49:0/4013731669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 msgr2=0x559531efd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f58098f0640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 0x7f57f00a4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cd0c0 tx=0x7f57fc086040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013620 tx=0x7f57f4009830 comp rx=0 tx=0).stop 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 -- 172.21.15.49:0/4013731669 shutdown_connections 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531f06650 0x559531f08a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f06110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 --2- 172.21.15.49:0/4013731669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531efd530 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 -- 172.21.15.49:0/4013731669 >> 172.21.15.49:0/4013731669 conn(0x559531ef7880 msgr2=0x559531ef7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 -- 172.21.15.49:0/4013731669 shutdown_connections 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.856+0000 7f580e018740 1 -- 172.21.15.49:0/4013731669 wait complete. 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 -- start start 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 Processor -- start 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531f03be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531f06650 0x559531f04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559531f09440 con 0x559531efd150 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559531e1d160 con 0x559531efda70 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559531f095c0 con 0x559531f06650 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f580e018740 10 client.0 _ll_register_callbacks cb 0x559531e01150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58778/0 (socket says 172.21.15.49:58778) 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f58088ee640 1 -- 172.21.15.49:0/3634267105 learned_addr learned my addr 172.21.15.49:0/3634267105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f58090ef640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531f03be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.857+0000 7f58088ee640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531f06650 0x559531f04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 -- 172.21.15.49:0/3634267105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531f06650 msgr2=0x559531f04120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531f06650 0x559531f04120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 -- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 msgr2=0x559531f03be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531f03be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 -- 172.21.15.49:0/3634267105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559531c3f7d0 con 0x559531efd150 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f58088ee640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f57f40057b0 tx=0x7f57f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2ad113e35e5b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f402e060 con 0x559531efd150 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f57f400f580 con 0x559531efd150 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f40041f0 con 0x559531efd150 2024-03-16T08:26:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f57f4024070 con 0x559531efd150 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 10 client.17811 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559531b753f0 con 0x559531efd150 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f40041f0 con 0x559531efd150 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f57f401e070 con 0x559531efd150 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 10 client.17811 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 20 client.17811 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f16600 0x559531f189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.858+0000 7f580e018740 4 client.17811 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.859+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559531e6ea40 con 0x559531f16600 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.859+0000 7f58090ef640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f16600 0x559531f189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.859+0000 7f58090ef640 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f16600 0x559531f189f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88aab043d98bd5bd server_cookie=976af435ffe4a078 in_seq=0 out_seq=0 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.859+0000 7f57ecff9640 10 client.17811 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f57ecff9640 1 -- 172.21.15.49:0/3634267105 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559531e6ea40 con 0x559531f16600 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f57ecff9640 10 client.17811 handle_command_reply: tid=0 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 client.17811 shutdown 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 msgr2=0x559531f036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f57f40057b0 tx=0x7f57f4005f90 comp rx=0 tx=0).stop 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 shutdown_connections 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f16600 0x559531f189f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531f06650 0x559531f04120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531f03be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f57ecff9640 0 client.17811 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 --2- 172.21.15.49:0/3634267105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531efd150 0x559531f036a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 >> 172.21.15.49:0/3634267105 conn(0x559531ef7880 msgr2=0x559531ef95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 shutdown_connections 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 1 -- 172.21.15.49:0/3634267105 wait complete. 2024-03-16T08:26:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.860+0000 7f580e018740 20 client.17811 trim_cache size 0 max 16384 2024-03-16T08:26:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 Processor -- start 2024-03-16T08:26:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 -- start start 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efd150 0x559531f02f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58041b1f80 con 0x559531f06650 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58041af850 con 0x559531efd150 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f580e018740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58041ad6c0 con 0x559531efda70 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58780/0 (socket says 172.21.15.49:58780) 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 -- 172.21.15.49:0/1560470069 learned_addr learned my addr 172.21.15.49:0/1560470069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efd150 0x559531f02f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58090ef640 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 -- 172.21.15.49:0/1560470069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 msgr2=0x559531f121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 -- 172.21.15.49:0/1560470069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efd150 msgr2=0x559531f02f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efd150 0x559531f02f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 -- 172.21.15.49:0/1560470069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559531b350f0 con 0x559531f06650 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.878+0000 7f58088ee640 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013570 tx=0x7f57f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aec9f535dee25e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f57eeffd640 1 -- 172.21.15.49:0/1560470069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f402e040 con 0x559531f06650 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f57eeffd640 1 -- 172.21.15.49:0/1560470069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4032a60 con 0x559531f06650 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f57eeffd640 1 -- 172.21.15.49:0/1560470069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f400f7a0 con 0x559531f06650 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 -- 172.21.15.49:0/1560470069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 msgr2=0x559531f1e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013570 tx=0x7f57f4009310 comp rx=0 tx=0).stop 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 -- 172.21.15.49:0/1560470069 shutdown_connections 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531f1e0e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efda70 0x559531f121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 --2- 172.21.15.49:0/1560470069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efd150 0x559531f02f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 -- 172.21.15.49:0/1560470069 >> 172.21.15.49:0/1560470069 conn(0x559531ef7880 msgr2=0x559531ef8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 -- 172.21.15.49:0/1560470069 shutdown_connections 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.879+0000 7f580e018740 1 -- 172.21.15.49:0/1560470069 wait complete. 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 -- start start 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 Processor -- start 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efd150 0x559531f00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531eff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58041add10 con 0x559531f06650 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58041af180 con 0x559531efda70 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58041ae3e0 con 0x559531efd150 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f580e018740 10 client.0 _ll_register_callbacks cb 0x7f57f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58088ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58788/0 (socket says 172.21.15.49:58788) 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58088ee640 1 -- 172.21.15.49:0/3664460757 learned_addr learned my addr 172.21.15.49:0/3664460757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58090ef640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531eff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58088ee640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efd150 0x559531f00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58090ef640 1 -- 172.21.15.49:0/3664460757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efd150 msgr2=0x559531f00f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58090ef640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efd150 0x559531f00f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.880+0000 7f58090ef640 1 -- 172.21.15.49:0/3664460757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 msgr2=0x559531effe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f58090ef640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f58090ef640 1 -- 172.21.15.49:0/3664460757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559531e289d0 con 0x559531efda70 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f58088ee640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f58090ef640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531eff910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5800009230 tx=0x7f5800005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f191e9e57ef5d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580004d070 con 0x559531efda70 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f4032a60 con 0x559531efda70 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f5800049b10 con 0x559531efda70 2024-03-16T08:26:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5800049d10 con 0x559531efda70 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f580e018740 10 client.27698 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559531f7be00 con 0x559531efda70 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4032a60 con 0x559531efda70 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.881+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f58000046c0 con 0x559531efda70 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 10 client.27698 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 10 client.27698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 20 client.27698 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f7c4d0 0x559531f7e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 4 client.27698 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559531e68d90 con 0x559531f7c4d0 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f58088ee640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f7c4d0 0x559531f7e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f57ecff9640 10 client.27698 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f58088ee640 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f7c4d0 0x559531f7e8f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4434412d74766c36 server_cookie=8d43a54673233653 in_seq=0 out_seq=0 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f57ecff9640 1 -- 172.21.15.49:0/3664460757 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559531e68d90 con 0x559531f7c4d0 2024-03-16T08:26:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f57ecff9640 10 client.27698 handle_command_reply: tid=0 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 client.27698 shutdown 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 msgr2=0x559531eff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531eff910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5800009230 tx=0x7f5800005fd0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 shutdown_connections 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x559531f7c4d0 0x559531f7e8f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559531f06650 0x559531effe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559531efda70 0x559531eff910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 --2- 172.21.15.49:0/3664460757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559531efd150 0x559531f00f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f57ecff9640 0 client.27698 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.882+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 >> 172.21.15.49:0/3664460757 conn(0x559531ef7880 msgr2=0x559531f03ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 shutdown_connections 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580e018740 1 -- 172.21.15.49:0/3664460757 wait complete. 2024-03-16T08:26:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580e018740 20 client.27698 trim_cache size 0 max 16384 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "88f70423-25a1-4e58-ac78-83a6d7ffea79": { 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "88f70423-25a1-4e58-ac78-83a6d7ffea79", 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:26:18.890 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 msgr2=0x7f57f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 0x7f57f00a4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cd0c0 tx=0x7f57fc086040 comp rx=0 tx=0).stop 2024-03-16T08:26:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 msgr2=0x7f580419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00eb10 tx=0x7f57fc00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 shutdown_connections 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f57f00a2970 0x7f57f00a4e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804107790 0x7f580419eb80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804102140 0x7f580419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 --2- 172.21.15.49:0/1330355972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5804101780 0x7f580419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 >> 172.21.15.49:0/1330355972 conn(0x7f58040fd210 msgr2=0x7f58040fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 shutdown_connections 2024-03-16T08:26:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:18.883+0000 7f580b353640 1 -- 172.21.15.49:0/1330355972 wait complete. 2024-03-16T08:26:18.894 INFO:tasks.cephfs.filesystem:scrub status for tag:88f70423-25a1-4e58-ac78-83a6d7ffea79 - {'path': '/', 'tag': '88f70423-25a1-4e58-ac78-83a6d7ffea79', 'options': 'recursive,force'} 2024-03-16T08:26:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:19 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:26:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:19 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:19 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:26:19.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:19 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:19.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:19 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:26:19.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:19 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:19.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:19 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:26:19.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:19 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:19.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:19 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:26:19.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:19 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:19.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:19 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:26:19.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:19 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:20 smithi154 ceph-mon[36471]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:20 smithi049 ceph-mon[31942]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:20.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:20 smithi129 ceph-mon[36561]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:22 smithi154 ceph-mon[36471]: pgmap v283: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:22 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:26:22.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:22 smithi049 ceph-mon[31942]: pgmap v283: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:22.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:22 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:26:22.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:22 smithi129 ceph-mon[36561]: pgmap v283: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:22.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:22 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:26:24.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:24 smithi049 ceph-mon[31942]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:24.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:24 smithi129 ceph-mon[36561]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:24.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:24 smithi154 ceph-mon[36471]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-16T08:26:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:26 smithi049 ceph-mon[31942]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:26:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:26 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:26:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:26 smithi129 ceph-mon[36561]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:26:26.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:26 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:26:26.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:26 smithi154 ceph-mon[36471]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-03-16T08:26:26.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:26 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:26:27.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:27 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:26:27.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:27 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:26:27.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:27 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:26:27.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:27 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:26:27.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:27 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:26:27.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:27 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:26:27.725 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:27 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:26:27.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:27 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:26:27.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:27 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:26:28.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:28 smithi129 ceph-mon[36561]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:28.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:28 smithi154 ceph-mon[36471]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:29.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:28 smithi049 ceph-mon[31942]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:30.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:30 smithi129 ceph-mon[36561]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:30.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:30 smithi154 ceph-mon[36471]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:30 smithi049 ceph-mon[31942]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:32.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:32 smithi129 ceph-mon[36561]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:32.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:32 smithi154 ceph-mon[36471]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:33.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:32 smithi049 ceph-mon[31942]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:34.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:34 smithi129 ceph-mon[36561]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:34.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:34 smithi154 ceph-mon[36471]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:35.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:34 smithi049 ceph-mon[31942]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:36.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:36 smithi129 ceph-mon[36561]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:36.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:36 smithi154 ceph-mon[36471]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:37.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:36 smithi049 ceph-mon[31942]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:38 smithi129 ceph-mon[36561]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:38.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:38 smithi154 ceph-mon[36471]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:39.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:38 smithi049 ceph-mon[31942]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:40.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:40 smithi129 ceph-mon[36561]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:40.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:40 smithi154 ceph-mon[36471]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:41.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:40 smithi049 ceph-mon[31942]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:42.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:42 smithi129 ceph-mon[36561]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:42.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:42 smithi154 ceph-mon[36471]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:43.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:42 smithi049 ceph-mon[31942]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:44.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:44 smithi129 ceph-mon[36561]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:44.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:44 smithi154 ceph-mon[36471]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:45.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:44 smithi049 ceph-mon[31942]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:46.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:46 smithi129 ceph-mon[36561]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:46.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:46 smithi154 ceph-mon[36471]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:47.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:46 smithi049 ceph-mon[31942]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:48.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:48 smithi129 ceph-mon[36561]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:48.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:26:48.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:26:48.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:48 smithi049 ceph-mon[31942]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:48.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:48 smithi154 ceph-mon[36471]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 -- 172.21.15.49:0/339309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 msgr2=0x7f98bc101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 --2- 172.21.15.49:0/339309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc101bf0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009930 tx=0x7f98a402bc30 comp rx=0 tx=0).stop 2024-03-16T08:26:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 -- 172.21.15.49:0/339309760 shutdown_connections 2024-03-16T08:26:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 --2- 172.21.15.49:0/339309760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc102130 0x7f98bc10e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 --2- 172.21.15.49:0/339309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc101bf0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 --2- 172.21.15.49:0/339309760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc107780 0x7f98bc107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 -- 172.21.15.49:0/339309760 >> 172.21.15.49:0/339309760 conn(0x7f98bc078be0 msgr2=0x7f98bc078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 -- 172.21.15.49:0/339309760 shutdown_connections 2024-03-16T08:26:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 -- 172.21.15.49:0/339309760 wait complete. 2024-03-16T08:26:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.019+0000 7f98c09c6640 1 Processor -- start 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 -- start start 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc102130 0x7f98bc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc107780 0x7f98bc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111b90 con 0x7f98bc101770 2024-03-16T08:26:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111a10 con 0x7f98bc102130 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98c09c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111d10 con 0x7f98bc107780 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42654/0 (socket says 172.21.15.49:42654) 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 -- 172.21.15.49:0/2136934072 learned_addr learned my addr 172.21.15.49:0/2136934072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc107780 0x7f98bc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc102130 0x7f98bc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 -- 172.21.15.49:0/2136934072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc107780 msgr2=0x7f98bc19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc107780 0x7f98bc19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 -- 172.21.15.49:0/2136934072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc102130 msgr2=0x7f98bc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc102130 0x7f98bc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 -- 172.21.15.49:0/2136934072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc19d080 con 0x7f98bc101770 2024-03-16T08:26:49.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.020+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00ea60 tx=0x7f98ac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.021+0000 7f98b37fe640 1 -- 172.21.15.49:0/2136934072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00cde0 con 0x7f98bc101770 2024-03-16T08:26:49.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.021+0000 7f98b37fe640 1 -- 172.21.15.49:0/2136934072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac0035f0 con 0x7f98bc101770 2024-03-16T08:26:49.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.021+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc19d370 con 0x7f98bc101770 2024-03-16T08:26:49.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.021+0000 7f98b37fe640 1 -- 172.21.15.49:0/2136934072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac010670 con 0x7f98bc101770 2024-03-16T08:26:49.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.021+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a4c00 con 0x7f98bc101770 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.023+0000 7f98b37fe640 1 -- 172.21.15.49:0/2136934072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f98ac0037a0 con 0x7f98bc101770 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.023+0000 7f98c35a8740 1 Processor -- start 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 -- start start 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b37fe640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 0x7f98a80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 0x563b55c57530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c60650 0x563b55c62a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b55bbb500 con 0x563b55c57150 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b55b63be0 con 0x563b55c57a70 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98c35a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563b55bb75a0 con 0x563b55c60650 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 0x7f98a80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b37fe640 1 -- 172.21.15.49:0/2136934072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f98ac0cf120 con 0x7f98bc101770 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40144/0 (socket says 172.21.15.49:40144) 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 -- 172.21.15.49:0/3982514813 learned_addr learned my addr 172.21.15.49:0/3982514813 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98ba59c640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 0x563b55c57530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98bad9d640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c60650 0x563b55c62a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 -- 172.21.15.49:0/3982514813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c60650 msgr2=0x563b55c62a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c60650 0x563b55c62a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 -- 172.21.15.49:0/3982514813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 msgr2=0x563b55c57530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 0x563b55c57530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98b3fff640 1 -- 172.21.15.49:0/3982514813 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b55b774b0 con 0x563b55c57a70 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.024+0000 7f98ba59c640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 0x563b55c57530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:49.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.025+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 0x7f98a80a4e30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98b4005fd0 tx=0x7f98b4011bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.025+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98a402cfd0 tx=0x7f98a40095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d238fafceceb6c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.025+0000 7f98b0ff9640 1 -- 172.21.15.49:0/3982514813 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4042070 con 0x563b55c57a70 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.025+0000 7f98b0ff9640 1 -- 172.21.15.49:0/3982514813 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a4004030 con 0x563b55c57a70 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.025+0000 7f98b0ff9640 1 -- 172.21.15.49:0/3982514813 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4004b10 con 0x563b55c57a70 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- 172.21.15.49:0/3982514813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 msgr2=0x563b55c60110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98a402cfd0 tx=0x7f98a40095f0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- 172.21.15.49:0/3982514813 shutdown_connections 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c60650 0x563b55c62a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57a70 0x563b55c60110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3982514813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57150 0x563b55c57530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- 172.21.15.49:0/3982514813 >> 172.21.15.49:0/3982514813 conn(0x563b55c51880 msgr2=0x563b55c51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- 172.21.15.49:0/3982514813 shutdown_connections 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- 172.21.15.49:0/3982514813 wait complete. 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- start start 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 Processor -- start 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 0x563b55c5d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c60650 0x563b55c5e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563b55c633e0 con 0x563b55c57a70 2024-03-16T08:26:49.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b55b77160 con 0x563b55c60650 2024-03-16T08:26:49.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563b55c63560 con 0x563b55c57150 2024-03-16T08:26:49.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.026+0000 7f98c35a8740 10 client.0 _ll_register_callbacks cb 0x563b55b5b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 0x563b55c5d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42672/0 (socket says 172.21.15.49:42672) 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98b3fff640 1 -- 172.21.15.49:0/3008487624 learned_addr learned my addr 172.21.15.49:0/3008487624 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98bad9d640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c60650 0x563b55c5e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 -- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c60650 msgr2=0x563b55c5e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c60650 0x563b55c5e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 -- 172.21.15.49:0/3008487624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 msgr2=0x563b55c5db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 -- 172.21.15.49:0/3008487624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b559997d0 con 0x563b55c57150 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98ba59c640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 0x563b55c5d650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0cc830 tx=0x7f98ac097b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee9b1252533d71d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0b4060 con 0x563b55c57150 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f98ac097db0 con 0x563b55c57150 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a4004030 con 0x563b55c57150 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f98ac09be40 con 0x563b55c57150 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98c35a8740 10 client.27712 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.027+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b558cf3f0 con 0x563b55c57150 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a4004030 con 0x563b55c57150 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f98ac09ab90 con 0x563b55c57150 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 10 client.27712 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 10 client.27712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 20 client.27712 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55c70550 0x563b55c729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 4 client.27712 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b55bc8a40 con 0x563b55c70550 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55c70550 0x563b55c729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98a1ffb640 10 client.27712 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.028+0000 7f98b3fff640 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55c70550 0x563b55c729e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede93b5a1ecdfc30 server_cookie=c4084524e5c9039e in_seq=0 out_seq=0 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98a1ffb640 1 -- 172.21.15.49:0/3008487624 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b55bc8a40 con 0x563b55c70550 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98a1ffb640 10 client.27712 handle_command_reply: tid=0 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 client.27712 shutdown 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 msgr2=0x563b55c5d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 0x563b55c5d650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0cc830 tx=0x7f98ac097b00 comp rx=0 tx=0).stop 2024-03-16T08:26:49.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 shutdown_connections 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55c70550 0x563b55c729e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c60650 0x563b55c5e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c57a70 0x563b55c5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98a1ffb640 0 client.27712 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 --2- 172.21.15.49:0/3008487624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57150 0x563b55c5d650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 >> 172.21.15.49:0/3008487624 conn(0x563b55c51880 msgr2=0x563b55c535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 shutdown_connections 2024-03-16T08:26:49.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 1 -- 172.21.15.49:0/3008487624 wait complete. 2024-03-16T08:26:49.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.029+0000 7f98c35a8740 20 client.27712 trim_cache size 0 max 16384 2024-03-16T08:26:49.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 Processor -- start 2024-03-16T08:26:49.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 -- start start 2024-03-16T08:26:49.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1afcc0 con 0x563b55c60650 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ad590 con 0x563b55c57150 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98c35a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ab400 con 0x563b55c57a70 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42674/0 (socket says 172.21.15.49:42674) 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98bad9d640 1 -- 172.21.15.49:0/2401338008 learned_addr learned my addr 172.21.15.49:0/2401338008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40156/0 (socket says 172.21.15.49:40156) 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.047+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 -- 172.21.15.49:0/2401338008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 msgr2=0x563b55c6c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c6c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 -- 172.21.15.49:0/2401338008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 msgr2=0x563b55c5fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 -- 172.21.15.49:0/2401338008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b55c5c600 con 0x563b55c60650 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f98b400fbd0 tx=0x7f98b4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8ad44fd4fbe53 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98a3fff640 1 -- 172.21.15.49:0/2401338008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b401a040 con 0x563b55c60650 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98a3fff640 1 -- 172.21.15.49:0/2401338008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b4018040 con 0x563b55c60650 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.048+0000 7f98a3fff640 1 -- 172.21.15.49:0/2401338008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40055c0 con 0x563b55c60650 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- 172.21.15.49:0/2401338008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 msgr2=0x563b55c50900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f98b400fbd0 tx=0x7f98b4009500 comp rx=0 tx=0).stop 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- 172.21.15.49:0/2401338008 shutdown_connections 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55c50900 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c6c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2401338008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- 172.21.15.49:0/2401338008 >> 172.21.15.49:0/2401338008 conn(0x563b55c51880 msgr2=0x563b55c51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- 172.21.15.49:0/2401338008 shutdown_connections 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- 172.21.15.49:0/2401338008 wait complete. 2024-03-16T08:26:49.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- start start 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 Processor -- start 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c64860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aba50 con 0x563b55c60650 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1acec0 con 0x563b55c57150 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ac120 con 0x563b55c57a70 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98c35a8740 10 client.0 _ll_register_callbacks cb 0x7f98ac00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42678/0 (socket says 172.21.15.49:42678) 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 -- 172.21.15.49:0/2028903944 learned_addr learned my addr 172.21.15.49:0/2028903944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.049+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c64860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 -- 172.21.15.49:0/2028903944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 msgr2=0x563b55c64da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c64da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 -- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 msgr2=0x563b55c64860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c64860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 -- 172.21.15.49:0/2028903944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5588f0f0 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98bad9d640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097e0 tx=0x7f98b4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5776489ccf5ab4b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b401a040 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f98b4020da0 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b4018040 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f98b4004a40 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98c35a8740 10 client.17835 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b55b829d0 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.050+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b4018040 con 0x563b55c60650 2024-03-16T08:26:49.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f98b403c070 con 0x563b55c60650 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 10 client.17835 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 20 client.17835 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55cd74d0 0x563b55cd98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 4 client.17835 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b55bc2d90 con 0x563b55cd74d0 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55cd74d0 0x563b55cd98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98ba59c640 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55cd74d0 0x563b55cd98f0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d317c74e2ff9112 server_cookie=ce448291aea196a0 in_seq=0 out_seq=0 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98a1ffb640 10 client.17835 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98a1ffb640 1 -- 172.21.15.49:0/2028903944 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b55bc2d90 con 0x563b55cd74d0 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98a1ffb640 10 client.17835 handle_command_reply: tid=0 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.051+0000 7f98c35a8740 1 client.17835 shutdown 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 msgr2=0x563b55cd5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097e0 tx=0x7f98b4005bd0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 shutdown_connections 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x563b55cd74d0 0x563b55cd98f0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b55c60650 0x563b55cd5930 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b55c57a70 0x563b55c64da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98a1ffb640 0 client.17835 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 --2- 172.21.15.49:0/2028903944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b55c57150 0x563b55c64860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 >> 172.21.15.49:0/2028903944 conn(0x563b55c51880 msgr2=0x563b55c53540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 shutdown_connections 2024-03-16T08:26:49.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 1 -- 172.21.15.49:0/2028903944 wait complete. 2024-03-16T08:26:49.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c35a8740 20 client.17835 trim_cache size 0 max 16384 2024-03-16T08:26:49.069 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:49.069 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:26:49.069 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:26:49.069 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:49.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 msgr2=0x7f98a80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 0x7f98a80a4e30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98b4005fd0 tx=0x7f98b4011bc0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 msgr2=0x7f98bc198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00ea60 tx=0x7f98ac00ef20 comp rx=0 tx=0).stop 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 shutdown_connections 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f98a80a2970 0x7f98a80a4e30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98bc107780 0x7f98bc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98bc102130 0x7f98bc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 --2- 172.21.15.49:0/2136934072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98bc101770 0x7f98bc198090 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 >> 172.21.15.49:0/2136934072 conn(0x7f98bc078be0 msgr2=0x7f98bc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 shutdown_connections 2024-03-16T08:26:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.052+0000 7f98c09c6640 1 -- 172.21.15.49:0/2136934072 wait complete. 2024-03-16T08:26:49.071 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:26:49.071 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:26:49.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 -- 172.21.15.49:0/440741746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0141077e0 msgr2=0x7fa014107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 --2- 172.21.15.49:0/440741746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0141077e0 0x7fa014107bc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 -- 172.21.15.49:0/440741746 shutdown_connections 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 --2- 172.21.15.49:0/440741746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa01406a830 0x7fa01410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 --2- 172.21.15.49:0/440741746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 0x7fa01406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 --2- 172.21.15.49:0/440741746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0141077e0 0x7fa014107bc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 -- 172.21.15.49:0/440741746 >> 172.21.15.49:0/440741746 conn(0x7fa014077780 msgr2=0x7fa014077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 -- 172.21.15.49:0/440741746 shutdown_connections 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 -- 172.21.15.49:0/440741746 wait complete. 2024-03-16T08:26:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.220+0000 7fa019ac0640 1 Processor -- start 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 -- start start 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 0x7fa014198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0141077e0 0x7fa01419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa014111ba0 con 0x7fa01406a830 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa014111a20 con 0x7fa014069e90 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa019ac0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa014111d20 con 0x7fa0141077e0 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42688/0 (socket says 172.21.15.49:42688) 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 -- 172.21.15.49:0/3081807138 learned_addr learned my addr 172.21.15.49:0/3081807138 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa0137fe640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 0x7fa014198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa013fff640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0141077e0 0x7fa01419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 -- 172.21.15.49:0/3081807138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0141077e0 msgr2=0x7fa01419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0141077e0 0x7fa01419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 -- 172.21.15.49:0/3081807138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 msgr2=0x7fa014198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 0x7fa014198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 -- 172.21.15.49:0/3081807138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01419d030 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.221+0000 7fa012ffd640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.222+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00400cd80 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.222+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa004010910 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.222+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004018990 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.222+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01419d320 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.222+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141a4bb0 con 0x7fa01406a830 2024-03-16T08:26:49.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.223+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014068a20 con 0x7fa01406a830 2024-03-16T08:26:49.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.224+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa004010ac0 con 0x7fa01406a830 2024-03-16T08:26:49.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.224+0000 7fa010ff9640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 0x7f9ffc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.225+0000 7fa0137fe640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 0x7f9ffc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.225+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fa0040cf560 con 0x7fa01406a830 2024-03-16T08:26:49.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.225+0000 7fa0137fe640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 0x7f9ffc0a4e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa00802efd0 tx=0x7fa00802cf90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.228+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa00409bc90 con 0x7fa01406a830 2024-03-16T08:26:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.453+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa014199390 con 0x7fa01406a830 2024-03-16T08:26:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.453+0000 7fa010ff9640 1 -- 172.21.15.49:0/3081807138 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fa014199390 con 0x7fa01406a830 2024-03-16T08:26:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:26:49.454 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:26:49.454 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 msgr2=0x7f9ffc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 0x7f9ffc0a4e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa00802efd0 tx=0x7fa00802cf90 comp rx=0 tx=0).stop 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 msgr2=0x7fa014198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 shutdown_connections 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f9ffc0a2970 0x7f9ffc0a4e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0141077e0 0x7fa01419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01406a830 0x7fa014198580 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 --2- 172.21.15.49:0/3081807138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa014069e90 0x7fa014198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 >> 172.21.15.49:0/3081807138 conn(0x7fa014077780 msgr2=0x7fa0140ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 shutdown_connections 2024-03-16T08:26:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.456+0000 7fa019ac0640 1 -- 172.21.15.49:0/3081807138 wait complete. 2024-03-16T08:26:49.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:26:49.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 -- 172.21.15.49:0/544061531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8101010 msgr2=0x7f11d810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 --2- 172.21.15.49:0/544061531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8101010 0x7f11d810d520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009930 tx=0x7f11c802bc30 comp rx=0 tx=0).stop 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 -- 172.21.15.49:0/544061531 shutdown_connections 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 --2- 172.21.15.49:0/544061531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8101010 0x7f11d810d520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 --2- 172.21.15.49:0/544061531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8100650 0x7f11d8100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 --2- 172.21.15.49:0/544061531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8106660 0x7f11d8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 -- 172.21.15.49:0/544061531 >> 172.21.15.49:0/544061531 conn(0x7f11d80777e0 msgr2=0x7f11d8077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 -- 172.21.15.49:0/544061531 shutdown_connections 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.577+0000 7f11df04d640 1 -- 172.21.15.49:0/544061531 wait complete. 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 Processor -- start 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 -- start start 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 0x7f11d8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8101010 0x7f11d81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8110a30 con 0x7f11d8106660 2024-03-16T08:26:49.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81108b0 con 0x7f11d8101010 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11df04d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8110bb0 con 0x7f11d8100650 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42698/0 (socket says 172.21.15.49:42698) 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dcde9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 0x7f11d8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 -- 172.21.15.49:0/4062064375 learned_addr learned my addr 172.21.15.49:0/4062064375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11d7fff640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8101010 0x7f11d81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 -- 172.21.15.49:0/4062064375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 msgr2=0x7f11d8102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 0x7f11d8102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 -- 172.21.15.49:0/4062064375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8101010 msgr2=0x7f11d81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8101010 0x7f11d81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 -- 172.21.15.49:0/4062064375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d81a6520 con 0x7f11d8106660 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dcde9640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 0x7f11d8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.578+0000 7f11dd5ea640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f11c8005940 tx=0x7f11c802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.579+0000 7f11d5ffb640 1 -- 172.21.15.49:0/4062064375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8041070 con 0x7f11d8106660 2024-03-16T08:26:49.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.579+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d81a67b0 con 0x7f11d8106660 2024-03-16T08:26:49.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.579+0000 7f11d5ffb640 1 -- 172.21.15.49:0/4062064375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c8004450 con 0x7f11d8106660 2024-03-16T08:26:49.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.579+0000 7f11d5ffb640 1 -- 172.21.15.49:0/4062064375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8003d30 con 0x7f11d8106660 2024-03-16T08:26:49.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.579+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a6cf0 con 0x7f11d8106660 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11d5ffb640 1 -- 172.21.15.49:0/4062064375 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f11c8044070 con 0x7f11d8106660 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 Processor -- start 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 -- start start 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed38120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56182ec90490 con 0x56182ed2d150 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182ec38c90 con 0x56182ed35d30 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11e1c30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56182ec8c530 con 0x56182ed2c830 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11dcde9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11dcde9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42708/0 (socket says 172.21.15.49:42708) 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11dcde9640 1 -- 172.21.15.49:0/3675868395 learned_addr learned my addr 172.21.15.49:0/3675868395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed38120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.582+0000 7f11d5ffb640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 0x7f11c00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 -- 172.21.15.49:0/3675868395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 msgr2=0x56182ed2cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dd5ea640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 0x7f11c00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed2cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 -- 172.21.15.49:0/3675868395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 msgr2=0x56182ed38120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed38120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 -- 172.21.15.49:0/3675868395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182ecf3500 con 0x56182ed2d150 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed2cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11d5ffb640 1 -- 172.21.15.49:0/4062064375 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f11c80f07a0 con 0x7f11d8106660 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f11d00117e0 tx=0x7f11d0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b74cb8fb3eff2a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11beffd640 1 -- 172.21.15.49:0/3675868395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0004d20 con 0x56182ed2d150 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11dd5ea640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 0x7f11c00a4e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11c80bcce0 tx=0x7f11c8009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11beffd640 1 -- 172.21.15.49:0/3675868395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d0005020 con 0x56182ed2d150 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.583+0000 7f11beffd640 1 -- 172.21.15.49:0/3675868395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0019670 con 0x56182ed2d150 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- 172.21.15.49:0/3675868395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 msgr2=0x56182ed357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f11d00117e0 tx=0x7f11d0011ca0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- 172.21.15.49:0/3675868395 shutdown_connections 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed38120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed357f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3675868395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed2cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- 172.21.15.49:0/3675868395 >> 172.21.15.49:0/3675868395 conn(0x56182ed26f60 msgr2=0x56182ed27340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- 172.21.15.49:0/3675868395 shutdown_connections 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- 172.21.15.49:0/3675868395 wait complete. 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 -- start start 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.584+0000 7f11e1c30740 1 Processor -- start 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2c830 0x56182ed32de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 0x56182ed33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56182ed38b20 con 0x56182ed2d150 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182ec4c230 con 0x56182ed2c830 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56182ed38ca0 con 0x56182ed35d30 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 10 client.0 _ll_register_callbacks cb 0x56182ec88010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42710/0 (socket says 172.21.15.49:42710) 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 -- 172.21.15.49:0/3996100947 learned_addr learned my addr 172.21.15.49:0/3996100947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2c830 0x56182ed32de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 0x56182ed33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 -- 172.21.15.49:0/3996100947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 msgr2=0x56182ed33860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 0x56182ed33860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 -- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2c830 msgr2=0x56182ed32de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2c830 0x56182ed32de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 -- 172.21.15.49:0/3996100947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182ec5b100 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 0x56182ed33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11dcde9640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f11d001b040 tx=0x7f11d0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21985ec8aec8655 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0028070 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f11d00053e0 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.585+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d0005020 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f11d0033050 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 10 client.17853 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56182e9a43f0 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d0005020 con 0x56182ed2d150 2024-03-16T08:26:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f11d0003b70 con 0x56182ed2d150 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 10 client.17853 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 10 client.17853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 20 client.17853 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182ed46140 0x56182ed48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 4 client.17853 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182ec9d9d0 con 0x56182ed46140 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.586+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182ed46140 0x56182ed48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11d7fff640 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182ed46140 0x56182ed48530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89e89ee1668daab0 server_cookie=732b968c98a5287e in_seq=0 out_seq=0 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11b7fff640 10 client.17853 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11b7fff640 1 -- 172.21.15.49:0/3996100947 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56182ec9d9d0 con 0x56182ed46140 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11b7fff640 10 client.17853 handle_command_reply: tid=0 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11e1c30740 1 client.17853 shutdown 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 msgr2=0x56182ed33320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f11d001b040 tx=0x7f11d0009880 comp rx=0 tx=0).stop 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 shutdown_connections 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.587+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182ed46140 0x56182ed48530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed35d30 0x56182ed33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2d150 0x56182ed33320 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11b7fff640 0 client.17853 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 --2- 172.21.15.49:0/3996100947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2c830 0x56182ed32de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 >> 172.21.15.49:0/3996100947 conn(0x56182ed26f60 msgr2=0x56182ed28cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 shutdown_connections 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 1 -- 172.21.15.49:0/3996100947 wait complete. 2024-03-16T08:26:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.588+0000 7f11e1c30740 20 client.17853 trim_cache size 0 max 16384 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 Processor -- start 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 -- start start 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed35510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 0x56182ed418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81b1fa0 con 0x56182ed35d30 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81af870 con 0x56182ed2d150 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11e1c30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81ad6e0 con 0x56182ed2c830 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42722/0 (socket says 172.21.15.49:42722) 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11d7fff640 1 -- 172.21.15.49:0/2823933342 learned_addr learned my addr 172.21.15.49:0/2823933342 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11d7fff640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed35510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11dcde9640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 0x56182ed418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.601+0000 7f11d7fff640 1 -- 172.21.15.49:0/2823933342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 msgr2=0x56182ed35510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11d7fff640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed35510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11d7fff640 1 -- 172.21.15.49:0/2823933342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 msgr2=0x56182ed418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11d7fff640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 0x56182ed418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11d7fff640 1 -- 172.21.15.49:0/2823933342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182ed22640 con 0x56182ed35d30 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11dcde9640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 0x56182ed418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11d7fff640 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f11cc004650 tx=0x7f11cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b207b529e7f19ff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11be7fc640 1 -- 172.21.15.49:0/2823933342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc011d50 con 0x56182ed35d30 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11be7fc640 1 -- 172.21.15.49:0/2823933342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11cc005500 con 0x56182ed35d30 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11be7fc640 1 -- 172.21.15.49:0/2823933342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0035f0 con 0x56182ed35d30 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11e1c30740 1 -- 172.21.15.49:0/2823933342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 msgr2=0x56182ed4dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.602+0000 7f11e1c30740 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f11cc004650 tx=0x7f11cc0046c0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- 172.21.15.49:0/2823933342 shutdown_connections 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed35d30 0x56182ed4dcc0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed2d150 0x56182ed418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- 172.21.15.49:0/2823933342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2c830 0x56182ed35510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- 172.21.15.49:0/2823933342 >> 172.21.15.49:0/2823933342 conn(0x56182ed26f60 msgr2=0x56182ed32da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- 172.21.15.49:0/2823933342 shutdown_connections 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- 172.21.15.49:0/2823933342 wait complete. 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- start start 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 Processor -- start 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2d150 0x56182ed2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed2ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81add30 con 0x56182ed2c830 2024-03-16T08:26:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81af1a0 con 0x56182ed35d30 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81ae400 con 0x56182ed2d150 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11e1c30740 10 client.0 _ll_register_callbacks cb 0x7f11cc009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42738/0 (socket says 172.21.15.49:42738) 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11d7fff640 1 -- 172.21.15.49:0/4135585550 learned_addr learned my addr 172.21.15.49:0/4135585550 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.603+0000 7f11dcde9640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2d150 0x56182ed2f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed2ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 -- 172.21.15.49:0/4135585550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2d150 msgr2=0x56182ed2f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2d150 0x56182ed2f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 -- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 msgr2=0x56182ed2ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed2ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 -- 172.21.15.49:0/4135585550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182ea6e7d0 con 0x56182ed2c830 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11d7fff640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00a010 tx=0x7f11cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45df033751533b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc038020 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f11cc0049a0 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11cc005500 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f11cc011b50 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11e1c30740 10 client.17859 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56182edac2c0 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11cc005500 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.604+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f11cc003850 con 0x56182ed2c830 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 10 client.17859 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 10 client.17859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 20 client.17859 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182edac950 0x56182edaed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 4 client.17859 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56182ec97d20 con 0x56182edac950 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11dcde9640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182edac950 0x56182edaed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11dcde9640 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182edac950 0x56182edaed70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1709829486ba9a3 server_cookie=c059a7f3fadf7cad in_seq=0 out_seq=0 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11b7fff640 10 client.17859 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11b7fff640 1 -- 172.21.15.49:0/4135585550 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56182ec97d20 con 0x56182edac950 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11b7fff640 10 client.17859 handle_command_reply: tid=0 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 client.17859 shutdown 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 msgr2=0x56182ed3e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00a010 tx=0x7f11cc005fd0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 shutdown_connections 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x56182edac950 0x56182edaed70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182ed35d30 0x56182ed2ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56182ed2d150 0x56182ed2f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11b7fff640 0 client.17859 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 --2- 172.21.15.49:0/4135585550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56182ed2c830 0x56182ed3e340 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.605+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 >> 172.21.15.49:0/4135585550 conn(0x56182ed26f60 msgr2=0x56182ed33610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 shutdown_connections 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11e1c30740 1 -- 172.21.15.49:0/4135585550 wait complete. 2024-03-16T08:26:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11e1c30740 20 client.17859 trim_cache size 0 max 16384 2024-03-16T08:26:49.613 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 msgr2=0x7f11c00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 0x7f11c00a4e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11c80bcce0 tx=0x7f11c8009260 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 msgr2=0x7f11d8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f11c8005940 tx=0x7f11c802cb30 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 shutdown_connections 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11c00a2970 0x7f11c00a4e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d8106660 0x7f11d8103600 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d8101010 0x7f11d81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 --2- 172.21.15.49:0/4062064375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11d8100650 0x7f11d8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 >> 172.21.15.49:0/4062064375 conn(0x7f11d80777e0 msgr2=0x7f11d8068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 shutdown_connections 2024-03-16T08:26:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.606+0000 7f11df04d640 1 -- 172.21.15.49:0/4062064375 wait complete. 2024-03-16T08:26:49.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:26:49.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 -- 172.21.15.49:0/1115936776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78106710 msgr2=0x7fab78106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 --2- 172.21.15.49:0/1115936776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78106710 0x7fab78106af0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fab74009930 tx=0x7fab7402bc30 comp rx=0 tx=0).stop 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 -- 172.21.15.49:0/1115936776 shutdown_connections 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 --2- 172.21.15.49:0/1115936776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab781010c0 0x7fab7810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 --2- 172.21.15.49:0/1115936776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab78100700 0x7fab78100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 --2- 172.21.15.49:0/1115936776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78106710 0x7fab78106af0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 -- 172.21.15.49:0/1115936776 >> 172.21.15.49:0/1115936776 conn(0x7fab78077840 msgr2=0x7fab78077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 -- 172.21.15.49:0/1115936776 shutdown_connections 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.728+0000 7fab8066e640 1 -- 172.21.15.49:0/1115936776 wait complete. 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 Processor -- start 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 -- start start 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab781010c0 0x7fab7819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab78106710 0x7fab7819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab78111c20 con 0x7fab78100700 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab78111aa0 con 0x7fab781010c0 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab8066e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab78111da0 con 0x7fab78106710 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42748/0 (socket says 172.21.15.49:42748) 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 -- 172.21.15.49:0/3961434723 learned_addr learned my addr 172.21.15.49:0/3961434723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7dc09640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab781010c0 0x7fab7819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab78106710 0x7fab7819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 -- 172.21.15.49:0/3961434723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab78106710 msgr2=0x7fab7819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab78106710 0x7fab7819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 -- 172.21.15.49:0/3961434723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab781010c0 msgr2=0x7fab7819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab781010c0 0x7fab7819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.729+0000 7fab7e40a640 1 -- 172.21.15.49:0/3961434723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7819f330 con 0x7fab78100700 2024-03-16T08:26:49.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab7e40a640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fab74009230 tx=0x7fab7402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab6b7fe640 1 -- 172.21.15.49:0/3961434723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74041070 con 0x7fab78100700 2024-03-16T08:26:49.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab6b7fe640 1 -- 172.21.15.49:0/3961434723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab74004450 con 0x7fab78100700 2024-03-16T08:26:49.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7819f5c0 con 0x7fab78100700 2024-03-16T08:26:49.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab6b7fe640 1 -- 172.21.15.49:0/3961434723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74003e50 con 0x7fab78100700 2024-03-16T08:26:49.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.730+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab781a6eb0 con 0x7fab78100700 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab6b7fe640 1 -- 172.21.15.49:0/3961434723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fab74044070 con 0x7fab78100700 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 Processor -- start 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 -- start start 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f271f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f27154490 con 0x561f271f1150 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f270fcc90 con 0x561f271f9d30 2024-03-16T08:26:49.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab83256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561f27150530 con 0x561f271f0830 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.733+0000 7fab7e40a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f271f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7dc09640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:60584/0 (socket says 172.21.15.49:60584) 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 -- 172.21.15.49:0/3934729110 learned_addr learned my addr 172.21.15.49:0/3934729110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab6b7fe640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 0x7fab640a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 -- 172.21.15.49:0/3934729110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 msgr2=0x561f271fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 -- 172.21.15.49:0/3934729110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 msgr2=0x561f271f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f271f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 -- 172.21.15.49:0/3934729110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f271b7500 con 0x561f271f0830 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab6b7fe640 1 -- 172.21.15.49:0/3961434723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fab740f05f0 con 0x7fab78100700 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 0x7fab640a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab70009f50 tx=0x7fab700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e44b22fa70bdba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab68bfe640 1 -- 172.21.15.49:0/3934729110 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70004250 con 0x561f271f0830 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab68bfe640 1 -- 172.21.15.49:0/3934729110 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7000fe80 con 0x561f271f0830 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.734+0000 7fab68bfe640 1 -- 172.21.15.49:0/3934729110 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7001bb20 con 0x561f271f0830 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 -- 172.21.15.49:0/3934729110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 msgr2=0x561f271f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 0x7fab640a4fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab70012fd0 tx=0x7fab70021000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab70009f50 tx=0x7fab700118d0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 -- 172.21.15.49:0/3934729110 shutdown_connections 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f271f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 --2- 172.21.15.49:0/3934729110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f0c10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 -- 172.21.15.49:0/3934729110 >> 172.21.15.49:0/3934729110 conn(0x561f271eaf60 msgr2=0x561f271eb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 -- 172.21.15.49:0/3934729110 shutdown_connections 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.735+0000 7fab83256740 1 -- 172.21.15.49:0/3934729110 wait complete. 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 -- start start 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 Processor -- start 2024-03-16T08:26:49.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 0x561f271f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f271fcb20 con 0x561f271f0830 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f27110230 con 0x561f271f9d30 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561f271fcca0 con 0x561f271f1150 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab83256740 10 client.0 _ll_register_callbacks cb 0x561f2714c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42760/0 (socket says 172.21.15.49:42760) 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 -- 172.21.15.49:0/1240486434 learned_addr learned my addr 172.21.15.49:0/1240486434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7dc09640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 0x561f271f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 -- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 msgr2=0x561f271f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 0x561f271f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 -- 172.21.15.49:0/1240486434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 msgr2=0x561f271f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7e40a640 1 -- 172.21.15.49:0/1240486434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2711f100 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.736+0000 7fab7dc09640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 0x561f271f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab7e40a640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fab740eded0 tx=0x7fab740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8330e3a4f0a14d29 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7404b070 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fab740c5090 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f271619d0 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab740bcc00 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 10 client.17871 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561f26e683f0 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561f271619d0 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fab740d5050 con 0x561f271f0830 2024-03-16T08:26:49.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 10 client.17871 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 10 client.17871 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 20 client.17871 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f2720a050 0x561f2720c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 4 client.17871 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.737+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f2715bd20 con 0x561f2720a050 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.738+0000 7fab7dc09640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f2720a050 0x561f2720c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.738+0000 7fab61ffb640 10 client.17871 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.738+0000 7fab7dc09640 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f2720a050 0x561f2720c510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aeaea02bda7ed9a server_cookie=4d2dcb6ebe57af34 in_seq=0 out_seq=0 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab61ffb640 1 -- 172.21.15.49:0/1240486434 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561f2715bd20 con 0x561f2720a050 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab61ffb640 10 client.17871 handle_command_reply: tid=0 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 client.17871 shutdown 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 msgr2=0x561f271f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fab740eded0 tx=0x7fab740057c0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 shutdown_connections 2024-03-16T08:26:49.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f2720a050 0x561f2720c510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f271f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f1150 0x561f271f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab61ffb640 0 client.17871 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 --2- 172.21.15.49:0/1240486434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f0830 0x561f271f6c90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 >> 172.21.15.49:0/1240486434 conn(0x561f271eaf60 msgr2=0x561f271eccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 shutdown_connections 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 1 -- 172.21.15.49:0/1240486434 wait complete. 2024-03-16T08:26:49.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.739+0000 7fab83256740 20 client.17871 trim_cache size 0 max 16384 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.752+0000 7fab83256740 1 Processor -- start 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.752+0000 7fab83256740 1 -- start start 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f27211e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab781b1f40 con 0x561f271f1150 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab781af810 con 0x561f271f9d30 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab83256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab781ad680 con 0x561f271f0830 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42768/0 (socket says 172.21.15.49:42768) 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 -- 172.21.15.49:0/2282052082 learned_addr learned my addr 172.21.15.49:0/2282052082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7e40a640 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f27211e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 -- 172.21.15.49:0/2282052082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 msgr2=0x561f271f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 -- 172.21.15.49:0/2282052082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 msgr2=0x561f27211e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f27211e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 -- 172.21.15.49:0/2282052082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f26f327d0 con 0x561f271f1150 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab7dc09640 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x561f271ec0f0 tx=0x7fab6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bccc605b2876270 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab63fff640 1 -- 172.21.15.49:0/2282052082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c028070 con 0x561f271f1150 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab63fff640 1 -- 172.21.15.49:0/2282052082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c0221b0 con 0x561f271f1150 2024-03-16T08:26:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.753+0000 7fab63fff640 1 -- 172.21.15.49:0/2282052082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00fb70 con 0x561f271f1150 2024-03-16T08:26:49.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- 172.21.15.49:0/2282052082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 msgr2=0x561f272058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x561f271ec0f0 tx=0x7fab6c005f90 comp rx=0 tx=0).stop 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- 172.21.15.49:0/2282052082 shutdown_connections 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f9d30 0x561f27211e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f1150 0x561f272058f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 --2- 172.21.15.49:0/2282052082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- 172.21.15.49:0/2282052082 >> 172.21.15.49:0/2282052082 conn(0x561f271eaf60 msgr2=0x561f271e9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- 172.21.15.49:0/2282052082 shutdown_connections 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- 172.21.15.49:0/2282052082 wait complete. 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 -- start start 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.754+0000 7fab83256740 1 Processor -- start 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 0x561f271f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab781adcd0 con 0x561f271f9d30 2024-03-16T08:26:49.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab781af140 con 0x561f271f1150 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab781ae3a0 con 0x561f271f0830 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab83256740 10 client.0 _ll_register_callbacks cb 0x7fab74005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42772/0 (socket says 172.21.15.49:42772) 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 -- 172.21.15.49:0/374189892 learned_addr learned my addr 172.21.15.49:0/374189892 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7e40a640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7dc09640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 0x561f271f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 -- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 msgr2=0x561f271f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 -- 172.21.15.49:0/374189892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 msgr2=0x561f271f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 0x561f271f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 -- 172.21.15.49:0/374189892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2726f5f0 con 0x561f271f9d30 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7dc09640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 0x561f271f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab7ec0b640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fab70011dd0 tx=0x7fab70009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9efab0ea407dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.755+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70035070 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7fab7000f920 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6c0221b0 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab7000fb20 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 10 client.17877 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561f27270890 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c0221b0 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fab7003a840 con 0x561f271f9d30 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 10 client.17877 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 10 client.17877 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 20 client.17877 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f27270e40 0x561f27273260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 4 client.17877 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f270d96a0 con 0x561f27270e40 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.756+0000 7fab7e40a640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f27270e40 0x561f27273260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab61ffb640 10 client.17877 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab7e40a640 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f27270e40 0x561f27273260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fca30eaa77cd461 server_cookie=c83da09beb039efb in_seq=0 out_seq=0 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab61ffb640 1 -- 172.21.15.49:0/374189892 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f270d96a0 con 0x561f27270e40 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab61ffb640 10 client.17877 handle_command_reply: tid=0 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 client.17877 shutdown 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 msgr2=0x561f271f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fab70011dd0 tx=0x7fab70009790 comp rx=0 tx=0).stop 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 shutdown_connections 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x561f27270e40 0x561f27273260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f271f9d30 0x561f271f4400 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab61ffb640 0 client.17877 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f271f1150 0x561f271f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 --2- 172.21.15.49:0/374189892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f271f0830 0x561f271f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 >> 172.21.15.49:0/374189892 conn(0x561f271eaf60 msgr2=0x561f271e9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 shutdown_connections 2024-03-16T08:26:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 1 -- 172.21.15.49:0/374189892 wait complete. 2024-03-16T08:26:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.757+0000 7fab83256740 20 client.17877 trim_cache size 0 max 16384 2024-03-16T08:26:49.764 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 msgr2=0x7fab640a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 0x7fab640a4fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab70012fd0 tx=0x7fab70021000 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 msgr2=0x7fab7819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fab74009230 tx=0x7fab7402cb30 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 shutdown_connections 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fab640a2b10 0x7fab640a4fd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab78106710 0x7fab7819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab781010c0 0x7fab7819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 --2- 172.21.15.49:0/3961434723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab78100700 0x7fab7819a340 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 >> 172.21.15.49:0/3961434723 conn(0x7fab78077840 msgr2=0x7fab7810b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 shutdown_connections 2024-03-16T08:26:49.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.758+0000 7fab8066e640 1 -- 172.21.15.49:0/3961434723 wait complete. 2024-03-16T08:26:49.767 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:49 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3081807138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:49.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:26:49.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 -- 172.21.15.49:0/2369251250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c102a50 msgr2=0x7f113c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 --2- 172.21.15.49:0/2369251250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c102a50 0x7f113c102e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1124009860 tx=0x7f112402bc00 comp rx=0 tx=0).stop 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 -- 172.21.15.49:0/2369251250 shutdown_connections 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 --2- 172.21.15.49:0/2369251250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 0x7f113c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 --2- 172.21.15.49:0/2369251250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c103370 0x7f113c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 --2- 172.21.15.49:0/2369251250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c102a50 0x7f113c102e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 -- 172.21.15.49:0/2369251250 >> 172.21.15.49:0/2369251250 conn(0x7f113c077840 msgr2=0x7f113c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 -- 172.21.15.49:0/2369251250 shutdown_connections 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.878+0000 7f1140db3640 1 -- 172.21.15.49:0/2369251250 wait complete. 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 Processor -- start 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 -- start start 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 0x7f113c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 0x7f113c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111bf0 con 0x7f113c102a50 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111a70 con 0x7f113c103370 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1140db3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111d70 con 0x7f113c10be20 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42778/0 (socket says 172.21.15.49:42778) 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 -- 172.21.15.49:0/2631254 learned_addr learned my addr 172.21.15.49:0/2631254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1139d9b640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 0x7f113c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 0x7f113c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 -- 172.21.15.49:0/2631254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 msgr2=0x7f113c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 0x7f113c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 -- 172.21.15.49:0/2631254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 msgr2=0x7f113c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 0x7f113c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f113a59c640 1 -- 172.21.15.49:0/2631254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c19d090 con 0x7f113c102a50 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.879+0000 7f1139d9b640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 0x7f113c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f113a59c640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1124037040 tx=0x7f11240344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f11337fe640 1 -- 172.21.15.49:0/2631254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124041020 con 0x7f113c102a50 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c19d320 con 0x7f113c102a50 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f11337fe640 1 -- 172.21.15.49:0/2631254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112402ca80 con 0x7f113c102a50 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f11337fe640 1 -- 172.21.15.49:0/2631254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124039040 con 0x7f113c102a50 2024-03-16T08:26:49.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.880+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c1a4c10 con 0x7f113c102a50 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.882+0000 7f114399a740 1 Processor -- start 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.882+0000 7f11337fe640 1 -- 172.21.15.49:0/2631254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f1124044020 con 0x7f113c102a50 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.882+0000 7f114399a740 1 -- start start 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a59e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a4f6490 con 0x55ed3a593150 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a49ec90 con 0x55ed3a59bd30 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f114399a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a4f2530 con 0x55ed3a592830 2024-03-16T08:26:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42784/0 (socket says 172.21.15.49:42784) 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:60652/0 (socket says 172.21.15.49:60652) 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113a59c640 1 -- 172.21.15.49:0/2683775774 learned_addr learned my addr 172.21.15.49:0/2683775774 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f1139d9b640 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a59e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f11337fe640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 0x7f11280a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 -- 172.21.15.49:0/2683775774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 msgr2=0x55ed3a59e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a59e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 -- 172.21.15.49:0/2683775774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 msgr2=0x55ed3a59b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 -- 172.21.15.49:0/2683775774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3a559500 con 0x55ed3a592830 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113a59c640 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f11337fe640 1 -- 172.21.15.49:0/2631254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f11240f0430 con 0x7f113c102a50 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 0x7f11280a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.883+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1134009fa0 tx=0x7f11340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd28610577553f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f1130ff9640 1 -- 172.21.15.49:0/2683775774 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134017050 con 0x55ed3a592830 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f113ad9d640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 0x7f11280a4e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1134022000 tx=0x7f1134023000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f1130ff9640 1 -- 172.21.15.49:0/2683775774 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134003b90 con 0x55ed3a592830 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f1130ff9640 1 -- 172.21.15.49:0/2683775774 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113401b970 con 0x55ed3a592830 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 -- 172.21.15.49:0/2683775774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 msgr2=0x55ed3a592c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1134009fa0 tx=0x7f11340118d0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 -- 172.21.15.49:0/2683775774 shutdown_connections 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a59e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a59b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 --2- 172.21.15.49:0/2683775774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a592c10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.884+0000 7f114399a740 1 -- 172.21.15.49:0/2683775774 >> 172.21.15.49:0/2683775774 conn(0x55ed3a58cf60 msgr2=0x55ed3a58d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- 172.21.15.49:0/2683775774 shutdown_connections 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- 172.21.15.49:0/2683775774 wait complete. 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- start start 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 Processor -- start 2024-03-16T08:26:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a598db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a599830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a59eb20 con 0x55ed3a593150 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a4b2230 con 0x55ed3a59bd30 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ed3a59eca0 con 0x55ed3a592830 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f114399a740 10 client.0 _ll_register_callbacks cb 0x55ed3a4ee010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f1139d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f113a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a598db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f1139d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42794/0 (socket says 172.21.15.49:42794) 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f1139d9b640 1 -- 172.21.15.49:0/168394607 learned_addr learned my addr 172.21.15.49:0/168394607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.885+0000 7f113ad9d640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a599830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 -- 172.21.15.49:0/168394607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 msgr2=0x55ed3a598db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a598db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 -- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 msgr2=0x55ed3a599830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a599830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 -- 172.21.15.49:0/168394607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3a4c1100 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1139d9b640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f112c011d10 tx=0x7f112c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53296245b7419623 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c004050 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f112c0041f0 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1134003b90 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f112c0043f0 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f114399a740 10 client.17889 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ed3a20a3f0 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.886+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134003b90 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f112c021020 con 0x55ed3a593150 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 10 client.17889 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 20 client.17889 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a5ac130 0x55ed3a5ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 4 client.17889 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed3a5039d0 con 0x55ed3a5ac130 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f113a59c640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a5ac130 0x55ed3a5ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f113a59c640 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a5ac130 0x55ed3a5ae5b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4d4251a348921e server_cookie=8540adfc9dbcd4b7 in_seq=0 out_seq=0 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.887+0000 7f1121ffb640 10 client.17889 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.888+0000 7f1121ffb640 1 -- 172.21.15.49:0/168394607 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed3a5039d0 con 0x55ed3a5ac130 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.888+0000 7f1121ffb640 10 client.17889 handle_command_reply: tid=0 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.888+0000 7f114399a740 1 client.17889 shutdown 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.888+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 msgr2=0x55ed3a5992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.888+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f112c011d10 tx=0x7f112c00f790 comp rx=0 tx=0).stop 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 shutdown_connections 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a5ac130 0x55ed3a5ae5b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a599830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f1121ffb640 0 client.17889 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a593150 0x55ed3a5992f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 --2- 172.21.15.49:0/168394607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a598db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 >> 172.21.15.49:0/168394607 conn(0x55ed3a58cf60 msgr2=0x55ed3a58ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 shutdown_connections 2024-03-16T08:26:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 1 -- 172.21.15.49:0/168394607 wait complete. 2024-03-16T08:26:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.889+0000 7f114399a740 20 client.17889 trim_cache size 0 max 16384 2024-03-16T08:26:49.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 Processor -- start 2024-03-16T08:26:49.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 -- start start 2024-03-16T08:26:49.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 0x55ed3a5a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a5b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1afd70 con 0x55ed3a592830 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ad640 con 0x55ed3a59bd30 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f114399a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ab4b0 con 0x55ed3a593150 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42806/0 (socket says 172.21.15.49:42806) 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 -- 172.21.15.49:0/3173321770 learned_addr learned my addr 172.21.15.49:0/3173321770 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113ad9d640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a5b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f1139d9b640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 0x55ed3a5a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 -- 172.21.15.49:0/3173321770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 msgr2=0x55ed3a5a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 0x55ed3a5a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 -- 172.21.15.49:0/3173321770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 msgr2=0x55ed3a5b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a5b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f113a59c640 1 -- 172.21.15.49:0/3173321770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3a597da0 con 0x55ed3a592830 2024-03-16T08:26:49.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.902+0000 7f1139d9b640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 0x55ed3a5a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f113ad9d640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a5b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f113a59c640 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11240bc8c0 tx=0x7f11240f5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b081387526a567 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f1123fff640 1 -- 172.21.15.49:0/3173321770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc990 con 0x55ed3a592830 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f1123fff640 1 -- 172.21.15.49:0/3173321770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11240bd2c0 con 0x55ed3a592830 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f1123fff640 1 -- 172.21.15.49:0/3173321770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc520 con 0x55ed3a592830 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 -- 172.21.15.49:0/3173321770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 msgr2=0x55ed3a59b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11240bc8c0 tx=0x7f11240f5040 comp rx=0 tx=0).stop 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 -- 172.21.15.49:0/3173321770 shutdown_connections 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a59bd30 0x55ed3a5b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a593150 0x55ed3a5a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 --2- 172.21.15.49:0/3173321770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a592830 0x55ed3a59b4e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 -- 172.21.15.49:0/3173321770 >> 172.21.15.49:0/3173321770 conn(0x55ed3a58cf60 msgr2=0x55ed3a58ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 -- 172.21.15.49:0/3173321770 shutdown_connections 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.903+0000 7f114399a740 1 -- 172.21.15.49:0/3173321770 wait complete. 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 -- start start 2024-03-16T08:26:49.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 Processor -- start 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a5a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a593150 0x55ed3a5a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1abb00 con 0x55ed3a59bd30 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1acf70 con 0x55ed3a593150 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ac1d0 con 0x55ed3a592830 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f114399a740 10 client.0 _ll_register_callbacks cb 0x7f112c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f1139d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a593150 0x55ed3a5a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42818/0 (socket says 172.21.15.49:42818) 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 -- 172.21.15.49:0/343867693 learned_addr learned my addr 172.21.15.49:0/343867693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a5a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 -- 172.21.15.49:0/343867693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 msgr2=0x55ed3a5a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a5a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 -- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a593150 msgr2=0x55ed3a5a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a593150 0x55ed3a5a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.904+0000 7f113ad9d640 1 -- 172.21.15.49:0/343867693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3a2d47d0 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f113ad9d640 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f113401b5e0 tx=0x7f113401b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab6bad79a70a29f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134036060 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f113401b680 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11240bd2c0 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1134003b90 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 10 client.17895 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ed3a612f60 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11240bd2c0 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f113401a070 con 0x55ed3a59bd30 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 10 client.17895 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:49.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 20 client.17895 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a6135a0 0x55ed3a6159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 4 client.17895 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.905+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed3a4fdd20 con 0x55ed3a6135a0 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f113a59c640 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a6135a0 0x55ed3a6159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f1121ffb640 10 client.17895 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f113a59c640 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a6135a0 0x55ed3a6159c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b035100d16900cb1 server_cookie=af6b36e51217ce52 in_seq=0 out_seq=0 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f1121ffb640 1 -- 172.21.15.49:0/343867693 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed3a4fdd20 con 0x55ed3a6135a0 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f1121ffb640 10 client.17895 handle_command_reply: tid=0 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 client.17895 shutdown 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 msgr2=0x55ed3a5a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f113401b5e0 tx=0x7f113401b570 comp rx=0 tx=0).stop 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 shutdown_connections 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55ed3a6135a0 0x55ed3a6159c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ed3a59bd30 0x55ed3a5a1430 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed3a593150 0x55ed3a5a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f1121ffb640 0 client.17895 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 --2- 172.21.15.49:0/343867693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed3a592830 0x55ed3a5a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 >> 172.21.15.49:0/343867693 conn(0x55ed3a58cf60 msgr2=0x55ed3a58e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 shutdown_connections 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 1 -- 172.21.15.49:0/343867693 wait complete. 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.906+0000 7f114399a740 20 client.17895 trim_cache size 0 max 16384 2024-03-16T08:26:49.913 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 msgr2=0x7f11280a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 0x7f11280a4e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1134022000 tx=0x7f1134023000 comp rx=0 tx=0).stop 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 msgr2=0x7f113c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1124037040 tx=0x7f11240344c0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 shutdown_connections 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f11280a2970 0x7f11280a4e30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113c10be20 0x7f113c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c103370 0x7f113c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 --2- 172.21.15.49:0/2631254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f113c102a50 0x7f113c1980f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:49.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 >> 172.21.15.49:0/2631254 conn(0x7f113c077840 msgr2=0x7f113c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:49.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 shutdown_connections 2024-03-16T08:26:49.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:49.907+0000 7f1140db3640 1 -- 172.21.15.49:0/2631254 wait complete. 2024-03-16T08:26:49.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:49 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3081807138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:50.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:49 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3081807138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:26:50.919 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T08:26:50.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:26:50.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T08:26:50.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:50 smithi049 ceph-mon[31942]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:50.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:50 smithi154 ceph-mon[36471]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:51.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 -- 172.21.15.49:0/1701209952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f752810e180 msgr2=0x7f7528110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 --2- 172.21.15.49:0/1701209952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f752810e180 0x7f7528110570 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7518009ca0 tx=0x7f751802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 -- 172.21.15.49:0/1701209952 shutdown_connections 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 --2- 172.21.15.49:0/1701209952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f752810e180 0x7f7528110570 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 --2- 172.21.15.49:0/1701209952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75280692e0 0x7f752810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 --2- 172.21.15.49:0/1701209952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 0x7f7528068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 -- 172.21.15.49:0/1701209952 >> 172.21.15.49:0/1701209952 conn(0x7f75280777e0 msgr2=0x7f7528077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 -- 172.21.15.49:0/1701209952 shutdown_connections 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 -- 172.21.15.49:0/1701209952 wait complete. 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.045+0000 7f752f33b640 1 Processor -- start 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 -- start start 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 0x7f752819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f752810e180 0x7f752819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7528113d40 con 0x7f75280692e0 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7528113bc0 con 0x7f75280689c0 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752f33b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7528113ec0 con 0x7f752810e180 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42840/0 (socket says 172.21.15.49:42840) 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 -- 172.21.15.49:0/839980432 learned_addr learned my addr 172.21.15.49:0/839980432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752d0d7640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 0x7f752819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752d8d8640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f752810e180 0x7f752819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 -- 172.21.15.49:0/839980432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f752810e180 msgr2=0x7f752819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f752810e180 0x7f752819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 -- 172.21.15.49:0/839980432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 msgr2=0x7f752819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 0x7f752819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 -- 172.21.15.49:0/839980432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752819f290 con 0x7f75280692e0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.046+0000 7f752c8d6640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9f0 tx=0x7f751c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.047+0000 7f75267fc640 1 -- 172.21.15.49:0/839980432 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c00cd80 con 0x7f75280692e0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.047+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752819f580 con 0x7f75280692e0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.047+0000 7f75267fc640 1 -- 172.21.15.49:0/839980432 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c005500 con 0x7f75280692e0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.047+0000 7f75267fc640 1 -- 172.21.15.49:0/839980432 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c010690 con 0x7f75280692e0 2024-03-16T08:26:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.047+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75281a6e10 con 0x7f75280692e0 2024-03-16T08:26:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.049+0000 7f75267fc640 1 -- 172.21.15.49:0/839980432 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f751c0046c0 con 0x7f75280692e0 2024-03-16T08:26:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 Processor -- start 2024-03-16T08:26:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 -- start start 2024-03-16T08:26:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f75267fc640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 0x7f75100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f752d8d8640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 0x7f75100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e678ae90d0 con 0x55e678b85a30 2024-03-16T08:26:51.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e678a91c70 con 0x55e678b86710 2024-03-16T08:26:51.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f7532000740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e678ae5170 con 0x55e678b8eda0 2024-03-16T08:26:51.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.050+0000 7f75267fc640 1 -- 172.21.15.49:0/839980432 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f751c0cf1c0 con 0x7f75280692e0 2024-03-16T08:26:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:40376/0 (socket says 172.21.15.49:40376) 2024-03-16T08:26:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 -- 172.21.15.49:0/1636799167 learned_addr learned my addr 172.21.15.49:0/1636799167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752c8d6640 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d8d8640 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 -- 172.21.15.49:0/1636799167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 msgr2=0x55e678b91190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b91190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 -- 172.21.15.49:0/1636799167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 msgr2=0x55e678b85eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b85eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 -- 172.21.15.49:0/1636799167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6788c57e0 con 0x55e678b86710 2024-03-16T08:26:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d8d8640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 0x7f75100a4e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7518005cc0 tx=0x7f751802ce60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7520009cf0 tx=0x7f7520011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7203bd40def103e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f750f7fe640 1 -- 172.21.15.49:0/1636799167 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752001e070 con 0x55e678b86710 2024-03-16T08:26:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f750f7fe640 1 -- 172.21.15.49:0/1636799167 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7520009de0 con 0x55e678b86710 2024-03-16T08:26:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.051+0000 7f750f7fe640 1 -- 172.21.15.49:0/1636799167 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75200196b0 con 0x55e678b86710 2024-03-16T08:26:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- 172.21.15.49:0/1636799167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 msgr2=0x55e678b85460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7520009cf0 tx=0x7f7520011810 comp rx=0 tx=0).stop 2024-03-16T08:26:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- 172.21.15.49:0/1636799167 shutdown_connections 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 --2- 172.21.15.49:0/1636799167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b85460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- 172.21.15.49:0/1636799167 >> 172.21.15.49:0/1636799167 conn(0x55e678b4cc50 msgr2=0x55e678b80480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- 172.21.15.49:0/1636799167 shutdown_connections 2024-03-16T08:26:51.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- 172.21.15.49:0/1636799167 wait complete. 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.052+0000 7f7532000740 1 -- start start 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 Processor -- start 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b8c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e678b91c10 con 0x55e678b85a30 2024-03-16T08:26:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e678aa50d0 con 0x55e678b86710 2024-03-16T08:26:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e678b91d90 con 0x55e678b8eda0 2024-03-16T08:26:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f7532000740 10 client.0 _ll_register_callbacks cb 0x55e678a6e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42852/0 (socket says 172.21.15.49:42852) 2024-03-16T08:26:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 -- 172.21.15.49:0/1161123383 learned_addr learned my addr 172.21.15.49:0/1161123383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752c8d6640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d8d8640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b8c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 -- 172.21.15.49:0/1161123383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 msgr2=0x55e678b8c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b8c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 -- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 msgr2=0x55e678b8c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752d0d7640 1 -- 172.21.15.49:0/1161123383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e678925f70 con 0x55e678b85a30 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.053+0000 7f752c8d6640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f752d0d7640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7520000c00 tx=0x7f75200056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aac21c7eb0cd17a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752001e040 con 0x55e678b85a30 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f75200035f0 con 0x55e678b85a30 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7520009de0 con 0x55e678b85a30 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7520030050 con 0x55e678b85a30 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f7532000740 10 client.17907 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e6787bd170 con 0x55e678b85a30 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7520009de0 con 0x55e678b85a30 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7520021070 con 0x55e678b85a30 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f7532000740 10 client.17907 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.054+0000 7f7532000740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f7532000740 20 client.17907 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678b9ed70 0x55e678ba1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f7532000740 4 client.17907 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e678af6610 con 0x55e678b9ed70 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f752c8d6640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678b9ed70 0x55e678ba1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f750cff9640 10 client.17907 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.055+0000 7f752c8d6640 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678b9ed70 0x55e678ba1160 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67eaffa34ac1494 server_cookie=bb9371daabe54b4 in_seq=0 out_seq=0 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f750cff9640 1 -- 172.21.15.49:0/1161123383 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e678af6610 con 0x55e678b9ed70 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f750cff9640 10 client.17907 handle_command_reply: tid=0 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 client.17907 shutdown 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 msgr2=0x55e678b8bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7520000c00 tx=0x7f75200056a0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 shutdown_connections 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678b9ed70 0x55e678ba1160 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b8c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 --2- 172.21.15.49:0/1161123383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b8bd00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f750cff9640 0 client.17907 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 >> 172.21.15.49:0/1161123383 conn(0x55e678b4cc50 msgr2=0x55e678b81940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 shutdown_connections 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 1 -- 172.21.15.49:0/1161123383 wait complete. 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.056+0000 7f7532000740 20 client.17907 trim_cache size 0 max 16384 2024-03-16T08:26:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 Processor -- start 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 -- start start 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b85a30 0x55e678b8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b86710 0x55e678b9aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75281b2000 con 0x55e678b8eda0 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75281af8d0 con 0x55e678b85a30 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f7532000740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75281ad740 con 0x55e678b86710 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42858/0 (socket says 172.21.15.49:42858) 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 -- 172.21.15.49:0/2566548942 learned_addr learned my addr 172.21.15.49:0/2566548942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752c8d6640 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b86710 0x55e678b9aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d0d7640 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b85a30 0x55e678b8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 -- 172.21.15.49:0/2566548942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b86710 msgr2=0x55e678b9aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b86710 0x55e678b9aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 -- 172.21.15.49:0/2566548942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b85a30 msgr2=0x55e678b8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.074+0000 7f752d8d8640 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b85a30 0x55e678b8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f752d8d8640 1 -- 172.21.15.49:0/2566548942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e678ba16f0 con 0x55e678b8eda0 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f752d8d8640 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f752806ad10 tx=0x7f751802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3168f5907e00a340 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f750effd640 1 -- 172.21.15.49:0/2566548942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518039040 con 0x55e678b8eda0 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f750effd640 1 -- 172.21.15.49:0/2566548942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7518052070 con 0x55e678b8eda0 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f750effd640 1 -- 172.21.15.49:0/2566548942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751804d450 con 0x55e678b8eda0 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 -- 172.21.15.49:0/2566548942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 msgr2=0x55e678b81dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f752806ad10 tx=0x7f751802e050 comp rx=0 tx=0).stop 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 -- 172.21.15.49:0/2566548942 shutdown_connections 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b8eda0 0x55e678b81dc0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b86710 0x55e678b9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 --2- 172.21.15.49:0/2566548942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b85a30 0x55e678b8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 -- 172.21.15.49:0/2566548942 >> 172.21.15.49:0/2566548942 conn(0x55e678b4cc50 msgr2=0x55e678b816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 -- 172.21.15.49:0/2566548942 shutdown_connections 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.075+0000 7f7532000740 1 -- 172.21.15.49:0/2566548942 wait complete. 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 -- start start 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 Processor -- start 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b90310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75281add90 con 0x55e678b85a30 2024-03-16T08:26:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75281af200 con 0x55e678b86710 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75281ae460 con 0x55e678b8eda0 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f7532000740 10 client.0 _ll_register_callbacks cb 0x7f752001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f752d0d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42866/0 (socket says 172.21.15.49:42866) 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f752d0d7640 1 -- 172.21.15.49:0/2403590153 learned_addr learned my addr 172.21.15.49:0/2403590153 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f752c8d6640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.076+0000 7f752d8d8640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 -- 172.21.15.49:0/2403590153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 msgr2=0x55e678b90310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b90310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 -- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 msgr2=0x55e678b8fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 -- 172.21.15.49:0/2403590153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e678b97610 con 0x55e678b85a30 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f752d0d7640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7520004830 tx=0x7f75200099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eed65842bcbd7a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752001e040 con 0x55e678b85a30 2024-03-16T08:26:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f7520004040 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7518052070 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7520009de0 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f7532000740 10 client.17913 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e678c058d0 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7518052070 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.077+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f752003c070 con 0x55e678b85a30 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 10 client.17913 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 10 client.17913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 20 client.17913 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678c06010 0x55e678c08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 4 client.17913 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e678b7bb40 con 0x55e678c06010 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f752c8d6640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678c06010 0x55e678c08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f752c8d6640 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678c06010 0x55e678c08430 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a839dd993d2212f9 server_cookie=a5716234d809f791 in_seq=0 out_seq=0 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.078+0000 7f750cff9640 10 client.17913 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f750cff9640 1 -- 172.21.15.49:0/2403590153 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e678b7bb40 con 0x55e678c06010 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f750cff9640 10 client.17913 handle_command_reply: tid=0 2024-03-16T08:26:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 client.17913 shutdown 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 msgr2=0x55e678b97050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7520004830 tx=0x7f75200099d0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 shutdown_connections 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55e678c06010 0x55e678c08430 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e678b8eda0 0x55e678b90310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e678b86710 0x55e678b8fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f750cff9640 0 client.17913 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 --2- 172.21.15.49:0/2403590153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e678b85a30 0x55e678b97050 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 >> 172.21.15.49:0/2403590153 conn(0x55e678b4cc50 msgr2=0x55e678b7f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 shutdown_connections 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 1 -- 172.21.15.49:0/2403590153 wait complete. 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.079+0000 7f7532000740 20 client.17913 trim_cache size 0 max 16384 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "6920923b-220c-49cc-a851-036419497977", 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-03-16T08:26:51.129 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 msgr2=0x7f75100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 0x7f75100a4e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7518005cc0 tx=0x7f751802ce60 comp rx=0 tx=0).stop 2024-03-16T08:26:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 msgr2=0x7f752819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9f0 tx=0x7f751c00edf0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 shutdown_connections 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75100a2970 0x7f75100a4e30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f752810e180 0x7f752819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75280692e0 0x7f752819a7e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 --2- 172.21.15.49:0/839980432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75280689c0 0x7f752819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 >> 172.21.15.49:0/839980432 conn(0x7f75280777e0 msgr2=0x7f752810f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 shutdown_connections 2024-03-16T08:26:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.080+0000 7f752f33b640 1 -- 172.21.15.49:0/839980432 wait complete. 2024-03-16T08:26:51.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:50 smithi129 ceph-mon[36561]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:26:51.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:26:51.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 -- 172.21.15.49:0/1966521242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c102a50 msgr2=0x7f5c3c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/1966521242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c102a50 0x7f5c3c102e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009900 tx=0x7f5c3802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 -- 172.21.15.49:0/1966521242 shutdown_connections 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/1966521242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c10be20 0x7f5c3c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/1966521242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/1966521242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c102a50 0x7f5c3c102e30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.285+0000 7f5c43c0b640 1 -- 172.21.15.49:0/1966521242 >> 172.21.15.49:0/1966521242 conn(0x7f5c3c077840 msgr2=0x7f5c3c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- 172.21.15.49:0/1966521242 shutdown_connections 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- 172.21.15.49:0/1966521242 wait complete. 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 Processor -- start 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- start start 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 0x7f5c3c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c111bf0 con 0x7f5c3c102a50 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c111a70 con 0x7f5c3c103370 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c43c0b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c111d70 con 0x7f5c3c10be20 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42898/0 (socket says 172.21.15.49:42898) 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c419a7640 1 -- 172.21.15.49:0/233503827 learned_addr learned my addr 172.21.15.49:0/233503827 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c411a6640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.286+0000 7f5c421a8640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 0x7f5c3c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 -- 172.21.15.49:0/233503827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 msgr2=0x7f5c3c101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 0x7f5c3c101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 -- 172.21.15.49:0/233503827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 msgr2=0x7f5c3c100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 -- 172.21.15.49:0/233503827 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c1a6590 con 0x7f5c3c102a50 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c411a6640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c421a8640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 0x7f5c3c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c419a7640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005940 tx=0x7f5c3802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c32ffd640 1 -- 172.21.15.49:0/233503827 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38041070 con 0x7f5c3c102a50 2024-03-16T08:26:51.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c1a6820 con 0x7f5c3c102a50 2024-03-16T08:26:51.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c32ffd640 1 -- 172.21.15.49:0/233503827 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38004450 con 0x7f5c3c102a50 2024-03-16T08:26:51.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c32ffd640 1 -- 172.21.15.49:0/233503827 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38003d80 con 0x7f5c3c102a50 2024-03-16T08:26:51.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.287+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c1a6d10 con 0x7f5c3c102a50 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.289+0000 7f5c32ffd640 1 -- 172.21.15.49:0/233503827 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5c38044070 con 0x7f5c3c102a50 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 Processor -- start 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 -- start start 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bda8c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 0x5563bda97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5563bd9f0500 con 0x5563bda8ca70 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563bd998be0 con 0x5563bda8c150 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.290+0000 7f5c467f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5563bd9ec5a0 con 0x5563bda95650 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c32ffd640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 0x7f5c280a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42910/0 (socket says 172.21.15.49:42910) 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 -- 172.21.15.49:0/634037227 learned_addr learned my addr 172.21.15.49:0/634037227 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c421a8640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bda8c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c411a6640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 0x5563bda97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c421a8640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 0x7f5c280a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c32ffd640 1 -- 172.21.15.49:0/233503827 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f5c380f0800 con 0x7f5c3c102a50 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 -- 172.21.15.49:0/634037227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 msgr2=0x5563bda97a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 0x5563bda97a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 -- 172.21.15.49:0/634037227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 msgr2=0x5563bda8c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bda8c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 -- 172.21.15.49:0/634037227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bd9ac4b0 con 0x5563bda8ca70 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c411a6640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 0x5563bda97a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c421a8640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bda8c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c419a7640 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5c380bcf70 tx=0x7f5c3802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785459b5f27081 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c421a8640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 0x7f5c280a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c340099e0 tx=0x7f5c34009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c27fff640 1 -- 172.21.15.49:0/634037227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3804b070 con 0x5563bda8ca70 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c27fff640 1 -- 172.21.15.49:0/634037227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38004c80 con 0x5563bda8ca70 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.291+0000 7f5c27fff640 1 -- 172.21.15.49:0/634037227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380b9650 con 0x5563bda8ca70 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 -- 172.21.15.49:0/634037227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 msgr2=0x5563bda95110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5c380bcf70 tx=0x7f5c3802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 -- 172.21.15.49:0/634037227 shutdown_connections 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda95650 0x5563bda97a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8ca70 0x5563bda95110 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 --2- 172.21.15.49:0/634037227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bda8c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 -- 172.21.15.49:0/634037227 >> 172.21.15.49:0/634037227 conn(0x5563bda86880 msgr2=0x5563bda86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 -- 172.21.15.49:0/634037227 shutdown_connections 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.292+0000 7f5c467f7740 1 -- 172.21.15.49:0/634037227 wait complete. 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 -- start start 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 Processor -- start 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 0x5563bda931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5563bd9f0500 con 0x5563bda8c150 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563bd9ac160 con 0x5563bda95650 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5563bda98200 con 0x5563bda8ca70 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c467f7740 10 client.0 _ll_register_callbacks cb 0x5563bd990150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42926/0 (socket says 172.21.15.49:42926) 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 -- 172.21.15.49:0/3510979431 learned_addr learned my addr 172.21.15.49:0/3510979431 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c421a8640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 0x5563bda931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c411a6640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 -- 172.21.15.49:0/3510979431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 msgr2=0x5563bda92c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 -- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 msgr2=0x5563bda931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 0x5563bda931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c419a7640 1 -- 172.21.15.49:0/3510979431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bd7ce7d0 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.293+0000 7f5c421a8640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 0x5563bda931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802c8c0 tx=0x7f5c38003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680ce3322bfdf56d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3804b070 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f5c380c5240 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c38004c80 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c380ee3a0 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 10 client.17925 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5563bd7043f0 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38004c80 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5c380bc5d0 con 0x5563bda8c150 2024-03-16T08:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 10 client.17925 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 20 client.17925 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdaa51f0 0x5563bdaa75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 4 client.17925 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563bd9fda40 con 0x5563bdaa51f0 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.294+0000 7f5c411a6640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdaa51f0 0x5563bdaa75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.295+0000 7f5c411a6640 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdaa51f0 0x5563bdaa75e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c85478632374f9 server_cookie=ce80290c4610a76c in_seq=0 out_seq=0 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.295+0000 7f5c257fa640 10 client.17925 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c257fa640 1 -- 172.21.15.49:0/3510979431 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5563bd9fda40 con 0x5563bdaa51f0 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c257fa640 10 client.17925 handle_command_reply: tid=0 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 client.17925 shutdown 2024-03-16T08:26:51.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 msgr2=0x5563bda92750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802c8c0 tx=0x7f5c38003820 comp rx=0 tx=0).stop 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 shutdown_connections 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdaa51f0 0x5563bdaa75e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda95650 0x5563bda931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c257fa640 0 client.17925 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3510979431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda8c150 0x5563bda92750 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 >> 172.21.15.49:0/3510979431 conn(0x5563bda86880 msgr2=0x5563bda885d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.296+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 shutdown_connections 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.297+0000 7f5c467f7740 1 -- 172.21.15.49:0/3510979431 wait complete. 2024-03-16T08:26:51.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.297+0000 7f5c467f7740 20 client.17925 trim_cache size 0 max 16384 2024-03-16T08:26:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 Processor -- start 2024-03-16T08:26:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 -- start start 2024-03-16T08:26:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bdaa0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1b1f30 con 0x5563bda95650 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1af800 con 0x5563bda8c150 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c467f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ad670 con 0x5563bda8ca70 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c421a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c419a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bdaa0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c421a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42936/0 (socket says 172.21.15.49:42936) 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.314+0000 7f5c421a8640 1 -- 172.21.15.49:0/2169393237 learned_addr learned my addr 172.21.15.49:0/2169393237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c411a6640 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 -- 172.21.15.49:0/2169393237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 msgr2=0x5563bda92ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 -- 172.21.15.49:0/2169393237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 msgr2=0x5563bda85900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 -- 172.21.15.49:0/2169393237 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bda8c8a0 con 0x5563bda8c150 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c421a8640 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c419a7640 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bdaa0e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5c380bcf70 tx=0x7f5c3802bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985f9e67f9b569e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c277fe640 1 -- 172.21.15.49:0/2169393237 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3804b070 con 0x5563bda8c150 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c277fe640 1 -- 172.21.15.49:0/2169393237 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38004c80 con 0x5563bda8c150 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.315+0000 7f5c277fe640 1 -- 172.21.15.49:0/2169393237 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380bcdb0 con 0x5563bda8c150 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- 172.21.15.49:0/2169393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 msgr2=0x5563bdaa0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bdaa0e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5c380bcf70 tx=0x7f5c3802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- 172.21.15.49:0/2169393237 shutdown_connections 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda85900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8ca70 0x5563bda92ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 --2- 172.21.15.49:0/2169393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8c150 0x5563bdaa0e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- 172.21.15.49:0/2169393237 >> 172.21.15.49:0/2169393237 conn(0x5563bda86880 msgr2=0x5563bda91690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- 172.21.15.49:0/2169393237 shutdown_connections 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- 172.21.15.49:0/2169393237 wait complete. 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 -- start start 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 Processor -- start 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.316+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 0x5563bda9d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8ca70 0x5563bda98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1adcc0 con 0x5563bda95650 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1af130 con 0x5563bda8ca70 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ae390 con 0x5563bda8c150 2024-03-16T08:26:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 10 client.0 _ll_register_callbacks cb 0x7f5c340116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42942/0 (socket says 172.21.15.49:42942) 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 -- 172.21.15.49:0/3589731123 learned_addr learned my addr 172.21.15.49:0/3589731123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 0x5563bda9d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c411a6640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8ca70 0x5563bda98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 -- 172.21.15.49:0/3589731123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 msgr2=0x5563bda9d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 0x5563bda9d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 -- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8ca70 msgr2=0x5563bda98a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8ca70 0x5563bda98a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 -- 172.21.15.49:0/3589731123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bd6c40f0 con 0x5563bda95650 2024-03-16T08:26:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 0x5563bda9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c421a8640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5c3401e950 tx=0x7f5c34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba40c044f92d957 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34030060 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8492+0+0 (secure 0 0 0) 0x7f5c3401db20 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.317+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c38004c80 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c3401dd20 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 10 client.17931 fetch_fsmap learned FSMap version 24 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5563bd9b79d0 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38004c80 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5c3401d070 con 0x5563bda95650 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 10 client.17931 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:26:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 20 client.17931 populate_metadata read hostname 'smithi049' 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdb0b770 0x5563bdb0db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 4 client.17931 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563bd9f7d90 con 0x5563bdb0b770 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdb0b770 0x5563bdb0db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c419a7640 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdb0b770 0x5563bdb0db90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17d6289b7ae3f9e server_cookie=2ace28505e3987fa in_seq=0 out_seq=0 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.318+0000 7f5c257fa640 10 client.17931 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c257fa640 1 -- 172.21.15.49:0/3589731123 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5563bd9f7d90 con 0x5563bdb0b770 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c257fa640 10 client.17931 handle_command_reply: tid=0 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 client.17931 shutdown 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 msgr2=0x5563bda98fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5c3401e950 tx=0x7f5c34004ab0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 shutdown_connections 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x5563bdb0b770 0x5563bdb0db90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563bda95650 0x5563bda98fa0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c257fa640 0 client.17931 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563bda8ca70 0x5563bda98a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 --2- 172.21.15.49:0/3589731123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5563bda8c150 0x5563bda9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 >> 172.21.15.49:0/3589731123 conn(0x5563bda86880 msgr2=0x5563bda85900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 shutdown_connections 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 1 -- 172.21.15.49:0/3589731123 wait complete. 2024-03-16T08:26:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c467f7740 20 client.17931 trim_cache size 0 max 16384 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "6920923b-220c-49cc-a851-036419497977": { 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "6920923b-220c-49cc-a851-036419497977", 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:26:51.328 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:26:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 msgr2=0x7f5c280a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 0x7f5c280a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c340099e0 tx=0x7f5c34009500 comp rx=0 tx=0).stop 2024-03-16T08:26:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 msgr2=0x7f5c3c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005940 tx=0x7f5c3802cbd0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 shutdown_connections 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5c280a29a0 0x7f5c280a4e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.319+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3c10be20 0x7f5c3c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.320+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c3c103370 0x7f5c3c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.320+0000 7f5c43c0b640 1 --2- 172.21.15.49:0/233503827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c102a50 0x7f5c3c1006a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.320+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 >> 172.21.15.49:0/233503827 conn(0x7f5c3c077840 msgr2=0x7f5c3c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.320+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 shutdown_connections 2024-03-16T08:26:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:26:51.320+0000 7f5c43c0b640 1 -- 172.21.15.49:0/233503827 wait complete. 2024-03-16T08:26:51.331 INFO:tasks.cephfs.filesystem:scrub status for tag:6920923b-220c-49cc-a851-036419497977 - {'path': '/', 'tag': '6920923b-220c-49cc-a851-036419497977', 'options': 'recursive,force'} 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: scrub queued for path: / 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: scrub summary: active paths [/] 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:52.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:52 smithi154 ceph-mon[36471]: scrub summary: idle 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: scrub queued for path: / 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: scrub summary: active paths [/] 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:53.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:52 smithi049 ceph-mon[31942]: scrub summary: idle 2024-03-16T08:26:53.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:53.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: scrub queued for path: / 2024-03-16T08:26:53.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:53.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: scrub summary: active paths [/] 2024-03-16T08:26:53.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: scrub summary: idle+waiting paths [/] 2024-03-16T08:26:53.111 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:52 smithi129 ceph-mon[36561]: scrub summary: idle 2024-03-16T08:26:53.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:53 smithi154 ceph-mon[36471]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:54.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:53 smithi049 ceph-mon[31942]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:54.110 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:53 smithi129 ceph-mon[36561]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:26:56.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:56 smithi129 ceph-mon[36561]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:56.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:56 smithi154 ceph-mon[36471]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:56.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:56 smithi049 ceph-mon[31942]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:26:58.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:26:58 smithi129 ceph-mon[36561]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:58.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:26:58 smithi154 ceph-mon[36471]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:26:58.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:26:58 smithi049 ceph-mon[31942]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:00.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:00 smithi129 ceph-mon[36561]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:27:00.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:00 smithi154 ceph-mon[36471]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:27:00.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:00 smithi049 ceph-mon[31942]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-16T08:27:02.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:02 smithi129 ceph-mon[36561]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:02.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:02 smithi154 ceph-mon[36471]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:02.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:02 smithi049 ceph-mon[31942]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:04.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:04 smithi129 ceph-mon[36561]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:04.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:04 smithi154 ceph-mon[36471]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:04.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:04 smithi049 ceph-mon[31942]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-16T08:27:06.340 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:06 smithi049 ceph-mon[31942]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-16T08:27:06.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:06 smithi129 ceph-mon[36561]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-16T08:27:06.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:06 smithi154 ceph-mon[36471]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-16T08:27:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:08 smithi129 ceph-mon[36561]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:27:08.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:08 smithi129 ceph-mon[36561]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T08:27:08.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:08 smithi154 ceph-mon[36471]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:27:08.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:08 smithi154 ceph-mon[36471]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T08:27:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:08 smithi049 ceph-mon[31942]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-16T08:27:08.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:08 smithi049 ceph-mon[31942]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T08:27:10.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:10 smithi129 ceph-mon[36561]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:10.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:10 smithi154 ceph-mon[36471]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:10.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:10 smithi049 ceph-mon[31942]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:12.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:12 smithi129 ceph-mon[36561]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:12.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:12 smithi154 ceph-mon[36471]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:12.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:12 smithi049 ceph-mon[31942]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:14.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:14 smithi129 ceph-mon[36561]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:14.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:14 smithi129 ceph-mon[36561]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T08:27:14.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:14 smithi154 ceph-mon[36471]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:14.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:14 smithi154 ceph-mon[36471]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T08:27:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:14 smithi049 ceph-mon[31942]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-16T08:27:14.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:14 smithi049 ceph-mon[31942]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T08:27:15.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 -- 172.21.15.49:0/2269288807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881006a0 msgr2=0x7f658810e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 --2- 172.21.15.49:0/2269288807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881006a0 0x7f658810e670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6580009cd0 tx=0x7f658002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 -- 172.21.15.49:0/2269288807 shutdown_connections 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 --2- 172.21.15.49:0/2269288807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881006a0 0x7f658810e670 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 --2- 172.21.15.49:0/2269288807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 0x7f6588100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 --2- 172.21.15.49:0/2269288807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881067c0 0x7f6588106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 -- 172.21.15.49:0/2269288807 >> 172.21.15.49:0/2269288807 conn(0x7f6588076ca0 msgr2=0x7f65880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:15.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 -- 172.21.15.49:0/2269288807 shutdown_connections 2024-03-16T08:27:15.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.608+0000 7f658ce97640 1 -- 172.21.15.49:0/2269288807 wait complete. 2024-03-16T08:27:15.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.609+0000 7f658ce97640 1 Processor -- start 2024-03-16T08:27:15.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.609+0000 7f658ce97640 1 -- start start 2024-03-16T08:27:15.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.609+0000 7f658ce97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 0x7f65881980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.609+0000 7f658ce97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881006a0 0x7f6588198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111cd0 con 0x7f65881067c0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111b50 con 0x7f65881006a0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111e50 con 0x7f65880ffce0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37266/0 (socket says 172.21.15.49:37266) 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 -- 172.21.15.49:0/1884589389 learned_addr learned my addr 172.21.15.49:0/1884589389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658659c640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 0x7f65881980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6585d9b640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881006a0 0x7f6588198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 -- 172.21.15.49:0/1884589389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 msgr2=0x7f65881980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 0x7f65881980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 -- 172.21.15.49:0/1884589389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881006a0 msgr2=0x7f6588198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881006a0 0x7f6588198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 -- 172.21.15.49:0/1884589389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658819d090 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f6586d9d640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f65800348f0 tx=0x7f6580032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580041020 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658819d320 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6580032e80 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580039040 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.610+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a4c10 con 0x7f65881067c0 2024-03-16T08:27:15.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.611+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658804f540 con 0x7f65881067c0 2024-03-16T08:27:15.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.613+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6580044020 con 0x7f65881067c0 2024-03-16T08:27:15.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.613+0000 7f65777fe640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 0x7f65700a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.613+0000 7f658659c640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 0x7f65700a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.613+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f65800f0d20 con 0x7f65881067c0 2024-03-16T08:27:15.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.614+0000 7f658659c640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 0x7f65700a4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65780095f0 tx=0x7f6578005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:15.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.617+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f658004b020 con 0x7f65881067c0 2024-03-16T08:27:15.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.822+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6588061c20 con 0x7f65700a2a10 2024-03-16T08:27:15.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.829+0000 7f65777fe640 1 -- 172.21.15.49:0/1884589389 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f6588061c20 con 0x7f65700a2a10 2024-03-16T08:27:15.830 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-16T00:00:00", "created": "2024-03-16T08:17:13", "first": "2024-03-16T08:18:00", "last": "2024-03-16T08:27:00", "last_pruned": "2024-03-16T08:27:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-16T08:27:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 msgr2=0x7f65700a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 0x7f65700a4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65780095f0 tx=0x7f6578005f90 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 msgr2=0x7f658819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f65800348f0 tx=0x7f6580032c90 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 shutdown_connections 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f65700a2a10 0x7f65700a4ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65881067c0 0x7f658819c960 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65881006a0 0x7f6588198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 --2- 172.21.15.49:0/1884589389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65880ffce0 0x7f65881980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 >> 172.21.15.49:0/1884589389 conn(0x7f6588076ca0 msgr2=0x7f6588104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 shutdown_connections 2024-03-16T08:27:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.831+0000 7f658ce97640 1 -- 172.21.15.49:0/1884589389 wait complete. 2024-03-16T08:27:15.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 -- 172.21.15.49:0/3677601226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7114102a50 msgr2=0x7f7114102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 --2- 172.21.15.49:0/3677601226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7114102a50 0x7f7114102e30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7104009900 tx=0x7f710402bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 -- 172.21.15.49:0/3677601226 shutdown_connections 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 --2- 172.21.15.49:0/3677601226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711410bf60 0x7f711410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 --2- 172.21.15.49:0/3677601226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114103400 0x7f711410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 --2- 172.21.15.49:0/3677601226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7114102a50 0x7f7114102e30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 -- 172.21.15.49:0/3677601226 >> 172.21.15.49:0/3677601226 conn(0x7f7114077800 msgr2=0x7f7114077c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 -- 172.21.15.49:0/3677601226 shutdown_connections 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.952+0000 7f71199c6640 1 -- 172.21.15.49:0/3677601226 wait complete. 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 Processor -- start 2024-03-16T08:27:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 -- start start 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114102a50 0x7f71141981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 0x7f711419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7114111d70 con 0x7f711410bf60 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7114111bf0 con 0x7f7114102a50 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71199c6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7114111ef0 con 0x7f711419cab0 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37284/0 (socket says 172.21.15.49:37284) 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 -- 172.21.15.49:0/4219634996 learned_addr learned my addr 172.21.15.49:0/4219634996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f7112ffd640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114102a50 0x7f71141981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71137fe640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 0x7f711419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 -- 172.21.15.49:0/4219634996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 msgr2=0x7f711419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 0x7f711419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 -- 172.21.15.49:0/4219634996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114102a50 msgr2=0x7f71141981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114102a50 0x7f71141981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f710bfff640 1 -- 172.21.15.49:0/4219634996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711419d5a0 con 0x7f711410bf60 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.953+0000 7f71137fe640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 0x7f711419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f710bfff640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9f0 tx=0x7f70fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00ccc0 con 0x7f711410bf60 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc010910 con 0x7f711410bf60 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc018920 con 0x7f711410bf60 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711419d890 con 0x7f711410bf60 2024-03-16T08:27:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.954+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141a5080 con 0x7f711410bf60 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.955+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711406a800 con 0x7f711410bf60 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.956+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f70fc010ac0 con 0x7f711410bf60 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.957+0000 7f71189c4640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 0x7f71000a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.957+0000 7f7112ffd640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 0x7f71000a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.957+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f70fc0cf1a0 con 0x7f711410bf60 2024-03-16T08:27:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.957+0000 7f7112ffd640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 0x7f71000a4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7104006010 tx=0x7f7104005dd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:15.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:15.961+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f70fc09bf20 con 0x7f711410bf60 2024-03-16T08:27:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.170+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 --> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7114103400 con 0x7f71000a2a10 2024-03-16T08:27:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.176+0000 7f71189c4640 1 -- 172.21.15.49:0/4219634996 <== mgr.15432 v2:172.21.15.49:6800/4265921640 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7114103400 con 0x7f71000a2a10 2024-03-16T08:27:16.177 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 msgr2=0x7f71000a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 0x7f71000a4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7104006010 tx=0x7f7104005dd0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 msgr2=0x7f7114198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9f0 tx=0x7f70fc00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 shutdown_connections 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f71000a2a10 0x7f71000a4ed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f711419cab0 0x7f711419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711410bf60 0x7f7114198730 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 --2- 172.21.15.49:0/4219634996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7114102a50 0x7f71141981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:16.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 >> 172.21.15.49:0/4219634996 conn(0x7f7114077800 msgr2=0x7f71140fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:16.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 shutdown_connections 2024-03-16T08:27:16.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:16.178+0000 7f71199c6640 1 -- 172.21.15.49:0/4219634996 wait complete. 2024-03-16T08:27:16.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:16 smithi049 ceph-mon[31942]: pgmap v312: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:27:16.190 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-16T08:27:16.201 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-16T08:27:16.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:16 smithi129 ceph-mon[36561]: pgmap v312: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:27:16.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:16 smithi154 ceph-mon[36471]: pgmap v312: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T08:27:17.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:17 smithi129 ceph-mon[36561]: from='client.17937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:17.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:17 smithi129 ceph-mon[36561]: from='client.17943 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:17.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:17 smithi154 ceph-mon[36471]: from='client.17937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:17.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:17 smithi154 ceph-mon[36471]: from='client.17943 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:17 smithi049 ceph-mon[31942]: from='client.17937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:17.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:17 smithi049 ceph-mon[31942]: from='client.17943 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:27:18.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:18 smithi129 ceph-mon[36561]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-16T08:27:18.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:18 smithi154 ceph-mon[36471]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-16T08:27:18.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:18 smithi049 ceph-mon[31942]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-16T08:27:20.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:20 smithi129 ceph-mon[36561]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:20.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:20 smithi154 ceph-mon[36471]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:20.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:20 smithi049 ceph-mon[31942]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:21.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T08:27:21.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 -- 172.21.15.49:0/380321434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 msgr2=0x7f685810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 --2- 172.21.15.49:0/380321434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f685810d520 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6848009cd0 tx=0x7f684802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 -- 172.21.15.49:0/380321434 shutdown_connections 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 --2- 172.21.15.49:0/380321434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f685810d520 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 --2- 172.21.15.49:0/380321434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858100650 0x7f6858100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 --2- 172.21.15.49:0/380321434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858106660 0x7f6858106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 -- 172.21.15.49:0/380321434 >> 172.21.15.49:0/380321434 conn(0x7f68580777e0 msgr2=0x7f6858077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 -- 172.21.15.49:0/380321434 shutdown_connections 2024-03-16T08:27:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.455+0000 7f685d9e6640 1 -- 172.21.15.49:0/380321434 wait complete. 2024-03-16T08:27:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 Processor -- start 2024-03-16T08:27:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 -- start start 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 0x7f6858198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858106660 0x7f685819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6858110a30 con 0x7f6858101010 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68581108b0 con 0x7f6858100650 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f685d9e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6858110bb0 con 0x7f6858106660 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f68567fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 0x7f6858198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f68577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858106660 0x7f685819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f68567fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37300/0 (socket says 172.21.15.49:37300) 2024-03-16T08:27:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.456+0000 7f68567fc640 1 -- 172.21.15.49:0/255966833 learned_addr learned my addr 172.21.15.49:0/255966833 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 -- 172.21.15.49:0/255966833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858106660 msgr2=0x7f685819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858106660 0x7f685819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 -- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 msgr2=0x7f6858198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 -- 172.21.15.49:0/255966833 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685819d010 con 0x7f6858100650 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f68567fc640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f6856ffd640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 0x7f6858198020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9f0 tx=0x7f685000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f685c9e4640 1 -- 172.21.15.49:0/255966833 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685000cd80 con 0x7f6858100650 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f685c9e4640 1 -- 172.21.15.49:0/255966833 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68500035f0 con 0x7f6858100650 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685819d300 con 0x7f6858100650 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f685c9e4640 1 -- 172.21.15.49:0/255966833 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850010690 con 0x7f6858100650 2024-03-16T08:27:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.457+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a4b90 con 0x7f6858100650 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.459+0000 7f685c9e4640 1 -- 172.21.15.49:0/255966833 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6850003b30 con 0x7f6858100650 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 Processor -- start 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 -- start start 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f685c9e4640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 0x7f68440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x562096256520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096256a60 0x562096258e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620961ad500 con 0x562096248e00 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562096155be0 con 0x562096249720 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68605d0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5620961a95a0 con 0x562096256a60 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68577fe640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 0x7f68440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f685c9e4640 1 -- 172.21.15.49:0/255966833 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f68500cf0f0 con 0x7f6858100650 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x562096256520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37312/0 (socket says 172.21.15.49:37312) 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f6856ffd640 1 -- 172.21.15.49:0/1660925485 learned_addr learned my addr 172.21.15.49:0/1660925485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68577fe640 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096256a60 0x562096258e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 -- 172.21.15.49:0/1660925485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096256a60 msgr2=0x562096258e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096256a60 0x562096258e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 -- 172.21.15.49:0/1660925485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 msgr2=0x5620962491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.460+0000 7f68567fc640 1 -- 172.21.15.49:0/1660925485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620961694b0 con 0x562096249720 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f6856ffd640 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f68577fe640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 0x7f68440a4e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6848005cc0 tx=0x7f6848032340 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f68567fc640 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x562096256520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68400114c0 tx=0x7f6840011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a47b49b8a278d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f684dffb640 1 -- 172.21.15.49:0/1660925485 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840004030 con 0x562096249720 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f684dffb640 1 -- 172.21.15.49:0/1660925485 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68400041d0 con 0x562096249720 2024-03-16T08:27:21.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f684dffb640 1 -- 172.21.15.49:0/1660925485 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684000f960 con 0x562096249720 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.461+0000 7f68605d0740 1 -- 172.21.15.49:0/1660925485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 msgr2=0x562096256520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x562096256520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68400114c0 tx=0x7f6840011980 comp rx=0 tx=0).stop 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- 172.21.15.49:0/1660925485 shutdown_connections 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096256a60 0x562096258e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x562096256520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- 172.21.15.49:0/1660925485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x5620962491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- 172.21.15.49:0/1660925485 >> 172.21.15.49:0/1660925485 conn(0x562096243880 msgr2=0x562096243c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- 172.21.15.49:0/1660925485 shutdown_connections 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- 172.21.15.49:0/1660925485 wait complete. 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- start start 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 Processor -- start 2024-03-16T08:27:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096249720 0x562096253f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096256a60 0x5620962544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620962598a0 con 0x562096248e00 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562096169160 con 0x562096256a60 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562096259a20 con 0x562096249720 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.462+0000 7f68605d0740 10 client.0 _ll_register_callbacks cb 0x56209614d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f68567fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096249720 0x562096253f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37322/0 (socket says 172.21.15.49:37322) 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f68577fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096256a60 0x5620962544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 -- 172.21.15.49:0/3606298514 learned_addr learned my addr 172.21.15.49:0/3606298514 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 -- 172.21.15.49:0/3606298514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096249720 msgr2=0x562096253f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096249720 0x562096253f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 -- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096256a60 msgr2=0x5620962544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096256a60 0x5620962544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 -- 172.21.15.49:0/3606298514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562095f8b7d0 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9c0 tx=0x7f6850009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4091558909c804 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685009f860 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f68500cca80 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68400041d0 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f68500a3eb0 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f68605d0740 10 client.17955 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.463+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562095ec13f0 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68400041d0 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6850097460 con 0x562096248e00 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 10 client.17955 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 20 client.17955 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096266b00 0x562096268f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 4 client.17955 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620961baa40 con 0x562096266b00 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68567fc640 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096266b00 0x562096268f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f6836ffd640 10 client.17955 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.464+0000 7f68567fc640 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096266b00 0x562096268f50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64de34b0dd5dcde server_cookie=4e7179dbbe29e302 in_seq=0 out_seq=0 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f6836ffd640 1 -- 172.21.15.49:0/3606298514 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5620961baa40 con 0x562096266b00 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f6836ffd640 10 client.17955 handle_command_reply: tid=0 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 client.17955 shutdown 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 msgr2=0x562096253a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9c0 tx=0x7f6850009510 comp rx=0 tx=0).stop 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 shutdown_connections 2024-03-16T08:27:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096266b00 0x562096268f50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096256a60 0x5620962544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096249720 0x562096253f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f6836ffd640 0 client.17955 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 --2- 172.21.15.49:0/3606298514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096248e00 0x562096253a20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 >> 172.21.15.49:0/3606298514 conn(0x562096243880 msgr2=0x5620962455d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.465+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 shutdown_connections 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.466+0000 7f68605d0740 1 -- 172.21.15.49:0/3606298514 wait complete. 2024-03-16T08:27:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.466+0000 7f68605d0740 20 client.17955 trim_cache size 0 max 16384 2024-03-16T08:27:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 Processor -- start 2024-03-16T08:27:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 -- start start 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x5620962626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209626e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68581adb80 con 0x562096243880 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68581ab450 con 0x562096249720 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.483+0000 7f68605d0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f68581a92c0 con 0x562096248e00 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37332/0 (socket says 172.21.15.49:37332) 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 -- 172.21.15.49:0/3220676085 learned_addr learned my addr 172.21.15.49:0/3220676085 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f68567fc640 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x5620962626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f68577fe640 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209626e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 -- 172.21.15.49:0/3220676085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 msgr2=0x5620962626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x5620962626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 -- 172.21.15.49:0/3220676085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 msgr2=0x56209626e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209626e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 -- 172.21.15.49:0/3220676085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562095e810f0 con 0x562096243880 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f6856ffd640 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f68500999c0 tx=0x7f685009b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e1172e95eedfae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f684d7fa640 1 -- 172.21.15.49:0/3220676085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500979b0 con 0x562096243880 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f684d7fa640 1 -- 172.21.15.49:0/3220676085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68500b4070 con 0x562096243880 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.484+0000 7f684d7fa640 1 -- 172.21.15.49:0/3220676085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850097550 con 0x562096243880 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- 172.21.15.49:0/3220676085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 msgr2=0x562096256150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f68500999c0 tx=0x7f685009b890 comp rx=0 tx=0).stop 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- 172.21.15.49:0/3220676085 shutdown_connections 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209626e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x5620962626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- 172.21.15.49:0/3220676085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x562096256150 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- 172.21.15.49:0/3220676085 >> 172.21.15.49:0/3220676085 conn(0x562096252660 msgr2=0x562096242900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- 172.21.15.49:0/3220676085 shutdown_connections 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- 172.21.15.49:0/3220676085 wait complete. 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 -- start start 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 Processor -- start 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.485+0000 7f68605d0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x56209625acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209625b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68581a9910 con 0x562096243880 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68581aad80 con 0x562096249720 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f68581a9fe0 con 0x562096248e00 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 10 client.0 _ll_register_callbacks cb 0x7f685009b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37340/0 (socket says 172.21.15.49:37340) 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 -- 172.21.15.49:0/1246839058 learned_addr learned my addr 172.21.15.49:0/1246839058 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68567fc640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x56209625acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68577fe640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209625b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 -- 172.21.15.49:0/1246839058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 msgr2=0x56209625acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x56209625acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 -- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 msgr2=0x56209625b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209625b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 -- 172.21.15.49:0/1246839058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620961749d0 con 0x562096243880 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6856ffd640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f685009bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa5cb82fd138de server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850097460 con 0x562096243880 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f68500cca80 con 0x562096243880 2024-03-16T08:27:21.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68500b4070 con 0x562096243880 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6850097600 con 0x562096243880 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 10 client.17961 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.486+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5620962cd150 con 0x562096243880 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68500b4070 con 0x562096243880 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f68500a0780 con 0x562096243880 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 10 client.17961 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 20 client.17961 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096256a60 0x5620962cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 4 client.17961 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620961b4d90 con 0x562096256a60 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.487+0000 7f68567fc640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096256a60 0x5620962cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68567fc640 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096256a60 0x5620962cf630 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02fa8acb119b8dd server_cookie=64ad4a4ec71f55b4 in_seq=0 out_seq=0 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f6836ffd640 10 client.17961 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:21.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f6836ffd640 1 -- 172.21.15.49:0/1246839058 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620961b4d90 con 0x562096256a60 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f6836ffd640 10 client.17961 handle_command_reply: tid=0 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 client.17961 shutdown 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 msgr2=0x56209625a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f685009bf50 comp rx=0 tx=0).stop 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 shutdown_connections 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x562096256a60 0x5620962cf630 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562096249720 0x56209625b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562096248e00 0x56209625acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f6836ffd640 0 client.17961 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 --2- 172.21.15.49:0/1246839058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562096243880 0x56209625a7b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 >> 172.21.15.49:0/1246839058 conn(0x562096252660 msgr2=0x562096242070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 shutdown_connections 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 1 -- 172.21.15.49:0/1246839058 wait complete. 2024-03-16T08:27:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.488+0000 7f68605d0740 20 client.17961 trim_cache size 0 max 16384 2024-03-16T08:27:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:21.497 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-03-16T08:27:21.497 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-03-16T08:27:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 msgr2=0x7f68440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 0x7f68440a4e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6848005cc0 tx=0x7f6848032340 comp rx=0 tx=0).stop 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 msgr2=0x7f6858198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 0x7f6858198020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9f0 tx=0x7f685000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 shutdown_connections 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f68440a2970 0x7f68440a4e30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6858106660 0x7f685819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6858101010 0x7f6858198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 --2- 172.21.15.49:0/255966833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6858100650 0x7f6858198020 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 >> 172.21.15.49:0/255966833 conn(0x7f68580777e0 msgr2=0x7f6858068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 shutdown_connections 2024-03-16T08:27:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.489+0000 7f685d9e6640 1 -- 172.21.15.49:0/255966833 wait complete. 2024-03-16T08:27:21.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T08:27:21.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:27:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 -- 172.21.15.49:0/782253085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 msgr2=0x7fa978107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 --2- 172.21.15.49:0/782253085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa978107b90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa960009930 tx=0x7fa96002bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 -- 172.21.15.49:0/782253085 shutdown_connections 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 --2- 172.21.15.49:0/782253085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa97810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 --2- 172.21.15.49:0/782253085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa978101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 --2- 172.21.15.49:0/782253085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa978107b90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 -- 172.21.15.49:0/782253085 >> 172.21.15.49:0/782253085 conn(0x7fa9780fd250 msgr2=0x7fa9780ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 -- 172.21.15.49:0/782253085 shutdown_connections 2024-03-16T08:27:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 -- 172.21.15.49:0/782253085 wait complete. 2024-03-16T08:27:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.608+0000 7fa97efa9640 1 Processor -- start 2024-03-16T08:27:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 -- start start 2024-03-16T08:27:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa9781980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111bc0 con 0x7fa9781077b0 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111a40 con 0x7fa978102160 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97efa9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111d40 con 0x7fa9781017a0 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97d546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97cd45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa9781980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42844/0 (socket says 172.21.15.49:42844) 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97d546640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37370/0 (socket says 172.21.15.49:37370) 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97cd45640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa9781980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.49:56396/0 (socket says 172.21.15.49:56396) 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 -- 172.21.15.49:0/1612779985 learned_addr learned my addr 172.21.15.49:0/1612779985 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 -- 172.21.15.49:0/1612779985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 msgr2=0x7fa9781980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa9781980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 -- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 msgr2=0x7fa97819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 -- 172.21.15.49:0/1612779985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819d0d0 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa97d546640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.609+0000 7fa977fff640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.610+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96800ccc0 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.610+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819d3c0 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.610+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96800ce60 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.610+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010690 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.610+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781a4c50 con 0x7fa978102160 2024-03-16T08:27:21.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.611+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978068a20 con 0x7fa978102160 2024-03-16T08:27:21.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.612+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa9680046c0 con 0x7fa978102160 2024-03-16T08:27:21.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.612+0000 7fa975ffb640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 0x7fa9640a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:21.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.613+0000 7fa97cd45640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 0x7fa9640a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:21.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.613+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7fa9680cf280 con 0x7fa978102160 2024-03-16T08:27:21.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.613+0000 7fa97cd45640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 0x7fa9640a4e30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa960005ab0 tx=0x7fa960005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:21.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.621+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa96809bfa0 con 0x7fa978102160 2024-03-16T08:27:21.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.844+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa978199430 con 0x7fa978102160 2024-03-16T08:27:21.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.845+0000 7fa975ffb640 1 -- 172.21.15.49:0/1612779985 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fa978199430 con 0x7fa978102160 2024-03-16T08:27:21.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:27:21.964 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:27:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:27:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 msgr2=0x7fa9640a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 0x7fa9640a4e30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa960005ab0 tx=0x7fa960005fd0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 msgr2=0x7fa978198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800edf0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 shutdown_connections 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fa9640a2970 0x7fa9640a4e30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9781077b0 0x7fa97819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102160 0x7fa978198620 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 --2- 172.21.15.49:0/1612779985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9781017a0 0x7fa9781980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 >> 172.21.15.49:0/1612779985 conn(0x7fa9780fd250 msgr2=0x7fa9780fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 shutdown_connections 2024-03-16T08:27:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:21.847+0000 7fa97efa9640 1 -- 172.21.15.49:0/1612779985 wait complete. 2024-03-16T08:27:21.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T08:27:22.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.077+0000 7f2b60b3b640 1 -- 172.21.15.49:0/2513320253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c101770 msgr2=0x7f2b5c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.077+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/2513320253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c101770 0x7f2b5c101bf0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009930 tx=0x7f2b4402bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:22.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 -- 172.21.15.49:0/2513320253 shutdown_connections 2024-03-16T08:27:22.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/2513320253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 0x7f2b5c10e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/2513320253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c101770 0x7f2b5c101bf0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/2513320253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c107780 0x7f2b5c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 -- 172.21.15.49:0/2513320253 >> 172.21.15.49:0/2513320253 conn(0x7f2b5c078be0 msgr2=0x7f2b5c078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 -- 172.21.15.49:0/2513320253 shutdown_connections 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 -- 172.21.15.49:0/2513320253 wait complete. 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 Processor -- start 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.078+0000 7f2b60b3b640 1 -- start start 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c101770 0x7f2b5c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 0x7f2b5c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c111b90 con 0x7f2b5c107780 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c111a10 con 0x7f2b5c102130 2024-03-16T08:27:22.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c111d10 con 0x7f2b5c101770 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37378/0 (socket says 172.21.15.49:37378) 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/77646110 learned_addr learned my addr 172.21.15.49:0/77646110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c101770 0x7f2b5c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 0x7f2b5c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/77646110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c101770 msgr2=0x7f2b5c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c101770 0x7f2b5c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/77646110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 msgr2=0x7f2b5c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 0x7f2b5c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/77646110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c19d080 con 0x7f2b5c107780 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00eb10 tx=0x7f2b4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b537fe640 1 -- 172.21.15.49:0/77646110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c004b20 con 0x7f2b5c107780 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b537fe640 1 -- 172.21.15.49:0/77646110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c004cc0 con 0x7f2b5c107780 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5c19d370 con 0x7f2b5c107780 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b537fe640 1 -- 172.21.15.49:0/77646110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c010760 con 0x7f2b5c107780 2024-03-16T08:27:22.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.079+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5c1a4c00 con 0x7f2b5c107780 2024-03-16T08:27:22.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.081+0000 7f2b537fe640 1 -- 172.21.15.49:0/77646110 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2b4c005020 con 0x7f2b5c107780 2024-03-16T08:27:22.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b63806740 1 Processor -- start 2024-03-16T08:27:22.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b537fe640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 0x7f2b480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 0x7f2b480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b537fe640 1 -- 172.21.15.49:0/77646110 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f2b4c0cf3c0 con 0x7f2b5c107780 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b63806740 1 -- start start 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b63806740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.082+0000 7f2b63806740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcece17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b63806740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcece4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b63806740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bcec3c490 con 0x562bcece1d30 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b63806740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562bcebe4c90 con 0x562bcecd8830 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b63806740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562bcec38530 con 0x562bcecd9150 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcece4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42872/0 (socket says 172.21.15.49:42872) 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3963394404 learned_addr learned my addr 172.21.15.49:0/3963394404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 0x7f2b480a4e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005ab0 tx=0x7f2b44005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcece17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3963394404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 msgr2=0x562bcece17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcece17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3963394404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 msgr2=0x562bcece4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcece4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3963394404 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bcec9f500 con 0x562bcecd8830 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcece4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009ca0 tx=0x7f2b54011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c99d3e4dbed6f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.083+0000 7f2b50ff9640 1 -- 172.21.15.49:0/3963394404 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401e070 con 0x562bcecd8830 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b50ff9640 1 -- 172.21.15.49:0/3963394404 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b54009e50 con 0x562bcecd8830 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b50ff9640 1 -- 172.21.15.49:0/3963394404 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400e9e0 con 0x562bcecd8830 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 -- 172.21.15.49:0/3963394404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 msgr2=0x562bcecd8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009ca0 tx=0x7f2b54011810 comp rx=0 tx=0).stop 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 -- 172.21.15.49:0/3963394404 shutdown_connections 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcece4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcece17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 --2- 172.21.15.49:0/3963394404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd8830 0x562bcecd8c10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 -- 172.21.15.49:0/3963394404 >> 172.21.15.49:0/3963394404 conn(0x562bcecd2f60 msgr2=0x562bcecd3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 -- 172.21.15.49:0/3963394404 shutdown_connections 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.084+0000 7f2b63806740 1 -- 172.21.15.49:0/3963394404 wait complete. 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 -- start start 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 Processor -- start 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcecdecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bcecdf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562bcece4b20 con 0x562bcece1d30 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562bcebf8230 con 0x562bcecd9150 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562bcece4ca0 con 0x562bcecd8830 2024-03-16T08:27:22.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b63806740 10 client.0 _ll_register_callbacks cb 0x562bcec34010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37398/0 (socket says 172.21.15.49:37398) 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/4160003818 learned_addr learned my addr 172.21.15.49:0/4160003818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bcecdf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcecdecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/4160003818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 msgr2=0x562bcecdecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcecdecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 msgr2=0x562bcecdf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bcecdf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.085+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/4160003818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bcec07100 con 0x562bcece1d30 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c09b9d0 tx=0x7f2b4c09b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6051413e6be140f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0b3020 con 0x562bcece1d30 2024-03-16T08:27:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f2b4c0a3f50 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b54009e50 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2b4c0ba2e0 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 10 client.17979 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562bce9503f0 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b54009e50 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2b4c0cf040 con 0x562bcece1d30 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 10 client.17979 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 10 client.17979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 20 client.17979 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bcecf20d0 0x562bcecf4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 4 client.17979 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.086+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bcec499d0 con 0x562bcecf20d0 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.087+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bcecf20d0 0x562bcecf4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.087+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bcecf20d0 0x562bcecf4500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2e584b4c9441214 server_cookie=dcd51985e1ac65be in_seq=0 out_seq=0 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.087+0000 7f2b41ffb640 10 client.17979 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b41ffb640 1 -- 172.21.15.49:0/4160003818 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562bcec499d0 con 0x562bcecf20d0 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b41ffb640 10 client.17979 handle_command_reply: tid=0 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 client.17979 shutdown 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 msgr2=0x562bcecdf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c09b9d0 tx=0x7f2b4c09b3a0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 shutdown_connections 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bcecf20d0 0x562bcecf4500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecdf730 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bcecdf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 --2- 172.21.15.49:0/4160003818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcecdecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 >> 172.21.15.49:0/4160003818 conn(0x562bcecd2f60 msgr2=0x562bcecd4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b41ffb640 0 client.17979 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:27:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 shutdown_connections 2024-03-16T08:27:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 1 -- 172.21.15.49:0/4160003818 wait complete. 2024-03-16T08:27:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.088+0000 7f2b63806740 20 client.17979 trim_cache size 0 max 16384 2024-03-16T08:27:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 Processor -- start 2024-03-16T08:27:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 -- start start 2024-03-16T08:27:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcece13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bceced8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1afc90 con 0x562bcece1d30 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ad560 con 0x562bcecd9150 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b63806740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ab3d0 con 0x562bcecd8830 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37404/0 (socket says 172.21.15.49:37404) 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/3455016542 learned_addr learned my addr 172.21.15.49:0/3455016542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bceced8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcece13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/3455016542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 msgr2=0x562bcece13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcece13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/3455016542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 msgr2=0x562bceced8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bceced8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 -- 172.21.15.49:0/3455016542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bcecce640 con 0x562bcece1d30 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bceced8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.102+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0a3cb0 tx=0x7f2b4c09bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff73351dce3538 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b43fff640 1 -- 172.21.15.49:0/3455016542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0b3070 con 0x562bcece1d30 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b43fff640 1 -- 172.21.15.49:0/3455016542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c09c140 con 0x562bcece1d30 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b43fff640 1 -- 172.21.15.49:0/3455016542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0a3fb0 con 0x562bcece1d30 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 -- 172.21.15.49:0/3455016542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 msgr2=0x562bcecd1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0a3cb0 tx=0x7f2b4c09bbc0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 -- 172.21.15.49:0/3455016542 shutdown_connections 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcece1d30 0x562bcecd1e60 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcecd9150 0x562bceced8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 --2- 172.21.15.49:0/3455016542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd8830 0x562bcece13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 -- 172.21.15.49:0/3455016542 >> 172.21.15.49:0/3455016542 conn(0x562bcecd2f60 msgr2=0x562bcecd1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 -- 172.21.15.49:0/3455016542 shutdown_connections 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.103+0000 7f2b63806740 1 -- 172.21.15.49:0/3455016542 wait complete. 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 -- start start 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 Processor -- start 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcecdb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcece1d30 0x562bcecdba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1aba20 con 0x562bcecd8830 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ace90 con 0x562bcece1d30 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ac0f0 con 0x562bcecd9150 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b63806740 10 client.0 _ll_register_callbacks cb 0x7f2b4c09b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37408/0 (socket says 172.21.15.49:37408) 2024-03-16T08:27:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3987306031 learned_addr learned my addr 172.21.15.49:0/3987306031 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcecdb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5ad9d640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcece1d30 0x562bcecdba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3987306031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 msgr2=0x562bcecdb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcecdb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcece1d30 msgr2=0x562bcecdba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcece1d30 0x562bcecdba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.104+0000 7f2b5a59c640 1 -- 172.21.15.49:0/3987306031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bcea1a7d0 con 0x562bcecd8830 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b5a59c640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009c40 tx=0x7f2b54009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e8c7e8fd96bff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401e040 con 0x562bcecd8830 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f2b540050d0 con 0x562bcecd8830 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4c09c140 con 0x562bcecd8830 2024-03-16T08:27:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2b5400e3b0 con 0x562bcecd8830 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 10 client.17985 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562bced581d0 con 0x562bcecd8830 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c09c140 con 0x562bcecd8830 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2b54003680 con 0x562bcecd8830 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 10 client.17985 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 10 client.17985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 20 client.17985 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bced586b0 0x562bced5aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 4 client.17985 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.105+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 --> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bcec43d20 con 0x562bced586b0 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bced586b0 0x562bced5aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b41ffb640 10 client.17985 ms_handle_connect on v2:172.21.15.129:6836/1811728297 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b59d9b640 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bced586b0 0x562bced5aaa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d98e42f2af6522e4 server_cookie=6705c0d8b95a3a00 in_seq=0 out_seq=0 2024-03-16T08:27:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b41ffb640 1 -- 172.21.15.49:0/3987306031 <== mds.2 v2:172.21.15.129:6836/1811728297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bcec43d20 con 0x562bced586b0 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b41ffb640 10 client.17985 handle_command_reply: tid=0 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 client.17985 shutdown 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 msgr2=0x562bcecea110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009c40 tx=0x7f2b54009860 comp rx=0 tx=0).stop 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 shutdown_connections 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:6836/1811728297,v1:172.21.15.129:6837/1811728297] conn(0x562bced586b0 0x562bced5aaa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562bcece1d30 0x562bcecdba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562bcecd9150 0x562bcecdb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 --2- 172.21.15.49:0/3987306031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562bcecd8830 0x562bcecea110 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b41ffb640 0 client.17985 ms_handle_reset on v2:172.21.15.129:6836/1811728297 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 >> 172.21.15.49:0/3987306031 conn(0x562bcecd2f60 msgr2=0x562bcecd1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 shutdown_connections 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 1 -- 172.21.15.49:0/3987306031 wait complete. 2024-03-16T08:27:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.106+0000 7f2b63806740 20 client.17985 trim_cache size 0 max 16384 2024-03-16T08:27:22.115 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 msgr2=0x7f2b480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 0x7f2b480a4e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005ab0 tx=0x7f2b44005fd0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 msgr2=0x7f2b5c19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00eb10 tx=0x7f2b4c00c2d0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 shutdown_connections 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f2b480a2970 0x7f2b480a4e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b5c107780 0x7f2b5c19c950 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b5c102130 0x7f2b5c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 --2- 172.21.15.49:0/77646110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5c101770 0x7f2b5c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 >> 172.21.15.49:0/77646110 conn(0x7f2b5c078be0 msgr2=0x7f2b5c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 shutdown_connections 2024-03-16T08:27:22.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.107+0000 7f2b60b3b640 1 -- 172.21.15.49:0/77646110 wait complete. 2024-03-16T08:27:22.118 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:22 smithi049 ceph-mon[31942]: pgmap v315: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:22.118 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:22 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1612779985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:27:22.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- 172.21.15.49:0/2562898847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 msgr2=0x7f55cc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 --2- 172.21.15.49:0/2562898847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc10e620 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009cd0 tx=0x7f55bc02c0f0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- 172.21.15.49:0/2562898847 shutdown_connections 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 --2- 172.21.15.49:0/2562898847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc10e620 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 --2- 172.21.15.49:0/2562898847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 0x7f55cc068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 --2- 172.21.15.49:0/2562898847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 0x7f55cc107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- 172.21.15.49:0/2562898847 >> 172.21.15.49:0/2562898847 conn(0x7f55cc0777e0 msgr2=0x7f55cc077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- 172.21.15.49:0/2562898847 shutdown_connections 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- 172.21.15.49:0/2562898847 wait complete. 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 Processor -- start 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.228+0000 7f55d19f4640 1 -- start start 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 0x7f55cc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 0x7f55cc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc111b30 con 0x7f55cc069380 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1119b0 con 0x7f55cc107760 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc111cb0 con 0x7f55cc0689c0 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37418/0 (socket says 172.21.15.49:37418) 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 -- 172.21.15.49:0/1336989290 learned_addr learned my addr 172.21.15.49:0/1336989290 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55caffd640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 0x7f55cc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 0x7f55cc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 -- 172.21.15.49:0/1336989290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 msgr2=0x7f55cc198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 0x7f55cc198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 -- 172.21.15.49:0/1336989290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 msgr2=0x7f55cc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 0x7f55cc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 -- 172.21.15.49:0/1336989290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc19d000 con 0x7f55cc069380 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e9f0 tx=0x7f55c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d09f2640 1 -- 172.21.15.49:0/1336989290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000cd80 con 0x7f55cc069380 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d09f2640 1 -- 172.21.15.49:0/1336989290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c0010910 con 0x7f55cc069380 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.229+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc19d2f0 con 0x7f55cc069380 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.230+0000 7f55d09f2640 1 -- 172.21.15.49:0/1336989290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0018990 con 0x7f55cc069380 2024-03-16T08:27:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.230+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a4b80 con 0x7f55cc069380 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d09f2640 1 -- 172.21.15.49:0/1336989290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f55c0010ac0 con 0x7f55cc069380 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 Processor -- start 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 -- start start 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 0x55e5b4c757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4bd0490 con 0x55e5b4c6c830 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4b78c90 con 0x55e5b4c6d150 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d45e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4bcc530 con 0x55e5b4c75d30 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55d09f2640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 0x7f55b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37428/0 (socket says 172.21.15.49:37428) 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55cb7fe640 1 -- 172.21.15.49:0/3810120910 learned_addr learned my addr 172.21.15.49:0/3810120910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55caffd640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 0x55e5b4c757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 0x7f55b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.232+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55d09f2640 1 -- 172.21.15.49:0/1336989290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f55c00cf0e0 con 0x7f55cc069380 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 -- 172.21.15.49:0/3810120910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 msgr2=0x55e5b4c78120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c78120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 -- 172.21.15.49:0/3810120910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 msgr2=0x55e5b4c757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 0x55e5b4c757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 -- 172.21.15.49:0/3810120910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b4c33500 con 0x55e5b4c6c830 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55caffd640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 0x55e5b4c757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f55bc004af0 tx=0x7f55bc032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad32f98926a9f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 0x7f55b40a4e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02c700 tx=0x7f55bc042000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55b1ffb640 1 -- 172.21.15.49:0/3810120910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc051070 con 0x55e5b4c6c830 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55b1ffb640 1 -- 172.21.15.49:0/3810120910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc032900 con 0x55e5b4c6c830 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.233+0000 7f55b1ffb640 1 -- 172.21.15.49:0/3810120910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc039040 con 0x55e5b4c6c830 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- 172.21.15.49:0/3810120910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 msgr2=0x55e5b4c6cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f55bc004af0 tx=0x7f55bc032590 comp rx=0 tx=0).stop 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- 172.21.15.49:0/3810120910 shutdown_connections 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c78120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6d150 0x55e5b4c757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3810120910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c6cc10 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- 172.21.15.49:0/3810120910 >> 172.21.15.49:0/3810120910 conn(0x55e5b4c66f60 msgr2=0x55e5b4c67340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- 172.21.15.49:0/3810120910 shutdown_connections 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- 172.21.15.49:0/3810120910 wait complete. 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 -- start start 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.234+0000 7f55d45e9740 1 Processor -- start 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4c78ac0 con 0x55e5b4c6c830 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4b8c230 con 0x55e5b4c75d30 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4c78c40 con 0x55e5b4c6d150 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 10 client.0 _ll_register_callbacks cb 0x55e5b4bc8010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37436/0 (socket says 172.21.15.49:37436) 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55ca7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 -- 172.21.15.49:0/3201736325 learned_addr learned my addr 172.21.15.49:0/3201736325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 -- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 msgr2=0x55e5b4c732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 -- 172.21.15.49:0/3201736325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 msgr2=0x55e5b4c73810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c73810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 -- 172.21.15.49:0/3201736325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b4b9b100 con 0x55e5b4c6c830 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55caffd640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f55c4012700 tx=0x7f55c4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f832726d8f400e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400f920 con 0x55e5b4c6c830 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f55c400fac0 con 0x55e5b4c6c830 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.235+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55bc032900 con 0x55e5b4c6c830 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55c400fcc0 con 0x55e5b4c6c830 2024-03-16T08:27:22.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 10 client.17997 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e5b48e43f0 con 0x55e5b4c6c830 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc032900 con 0x55e5b4c6c830 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f55c4023070 con 0x55e5b4c6c830 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 10 client.17997 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 10 client.17997 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 20 client.17997 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4c86130 0x55e5b4c88580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 4 client.17997 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5b4bdd9d0 con 0x55e5b4c86130 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.236+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4c86130 0x55e5b4c88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.237+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4c86130 0x55e5b4c88580 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90667502ab8079e2 server_cookie=ee8de36d90688cfc in_seq=0 out_seq=0 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.237+0000 7f55aaffd640 10 client.17997 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.237+0000 7f55aaffd640 1 -- 172.21.15.49:0/3201736325 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5b4bdd9d0 con 0x55e5b4c86130 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.237+0000 7f55aaffd640 10 client.17997 handle_command_reply: tid=0 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.237+0000 7f55d45e9740 1 client.17997 shutdown 2024-03-16T08:27:22.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 msgr2=0x55e5b4c72d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f55c4012700 tx=0x7f55c4012bc0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 shutdown_connections 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4c86130 0x55e5b4c88580 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c73810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55aaffd640 0 client.17997 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3201736325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c72d90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 >> 172.21.15.49:0/3201736325 conn(0x55e5b4c66f60 msgr2=0x55e5b4c68cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 shutdown_connections 2024-03-16T08:27:22.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 1 -- 172.21.15.49:0/3201736325 wait complete. 2024-03-16T08:27:22.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.238+0000 7f55d45e9740 20 client.17997 trim_cache size 0 max 16384 2024-03-16T08:27:22.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 Processor -- start 2024-03-16T08:27:22.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 -- start start 2024-03-16T08:27:22.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c81860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c8de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1afd80 con 0x55e5b4c6c830 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ad650 con 0x55e5b4c75d30 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55d45e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ab4c0 con 0x55e5b4c6d150 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37444/0 (socket says 172.21.15.49:37444) 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55caffd640 1 -- 172.21.15.49:0/3987215950 learned_addr learned my addr 172.21.15.49:0/3987215950 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c81860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.251+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c8de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 -- 172.21.15.49:0/3987215950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 msgr2=0x55e5b4c81860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c81860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 -- 172.21.15.49:0/3987215950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 msgr2=0x55e5b4c8de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c8de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 -- 172.21.15.49:0/3987215950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b49ae7d0 con 0x55e5b4c6c830 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c8de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55caffd640 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009970 tx=0x7f55c4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3398b7c412b7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55b17fa640 1 -- 172.21.15.49:0/3987215950 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4029030 con 0x55e5b4c6c830 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55b17fa640 1 -- 172.21.15.49:0/3987215950 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c401c6e0 con 0x55e5b4c6c830 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55b17fa640 1 -- 172.21.15.49:0/3987215950 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40053c0 con 0x55e5b4c6c830 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55d45e9740 1 -- 172.21.15.49:0/3987215950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 msgr2=0x55e5b4c754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.252+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009970 tx=0x7f55c4009e60 comp rx=0 tx=0).stop 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- 172.21.15.49:0/3987215950 shutdown_connections 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c75d30 0x55e5b4c8de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c6d150 0x55e5b4c81860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- 172.21.15.49:0/3987215950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6c830 0x55e5b4c754c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- 172.21.15.49:0/3987215950 >> 172.21.15.49:0/3987215950 conn(0x55e5b4c66f60 msgr2=0x55e5b4c72d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- 172.21.15.49:0/3987215950 shutdown_connections 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- 172.21.15.49:0/3987215950 wait complete. 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- start start 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 Processor -- start 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1abb10 con 0x55e5b4c6d150 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1acf80 con 0x55e5b4c6c830 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ac1e0 con 0x55e5b4c75d30 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55d45e9740 10 client.0 _ll_register_callbacks cb 0x7f55c4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55caffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.253+0000 7f55ca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:42936/0 (socket says 172.21.15.49:42936) 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55ca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37456/0 (socket says 172.21.15.49:37456) 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 -- 172.21.15.49:0/1218680464 learned_addr learned my addr 172.21.15.49:0/1218680464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55cb7fe640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 -- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 msgr2=0x55e5b4c7a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c7a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 -- 172.21.15.49:0/1218680464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 msgr2=0x55e5b4c79cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 -- 172.21.15.49:0/1218680464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b4ceb4b0 con 0x55e5b4c6c830 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55caffd640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55c401c590 tx=0x7f55c4005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68bf9693d170b4d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4029040 con 0x55e5b4c6c830 2024-03-16T08:27:22.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c401c6e0 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f55c401c890 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f55c4005710 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55d45e9740 10 client.27884 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.254+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e5b4cec790 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c401c6e0 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f55c4034020 con 0x55e5b4c6c830 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 10 client.27884 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 10 client.27884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 20 client.27884 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4cecde0 0x55e5b4cef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 4 client.27884 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 --> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5b4bd7d20 con 0x55e5b4cecde0 2024-03-16T08:27:22.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4cecde0 0x55e5b4cef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55aaffd640 10 client.27884 ms_handle_connect on v2:172.21.15.154:6838/1693946357 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55ca7fc640 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4cecde0 0x55e5b4cef200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e35c21a7ddf74dd server_cookie=f68ccc09511fa648 in_seq=0 out_seq=0 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55aaffd640 1 -- 172.21.15.49:0/1218680464 <== mds.1 v2:172.21.15.154:6838/1693946357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5b4bd7d20 con 0x55e5b4cecde0 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55aaffd640 10 client.27884 handle_command_reply: tid=0 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.255+0000 7f55d45e9740 1 client.27884 shutdown 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 msgr2=0x55e5b4c797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55c401c590 tx=0x7f55c4005430 comp rx=0 tx=0).stop 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 shutdown_connections 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:6838/1693946357,v1:172.21.15.154:6839/1693946357] conn(0x55e5b4cecde0 0x55e5b4cef200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5b4c75d30 0x55e5b4c7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55aaffd640 0 client.27884 ms_handle_reset on v2:172.21.15.154:6838/1693946357 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5b4c6d150 0x55e5b4c79cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 --2- 172.21.15.49:0/1218680464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5b4c6c830 0x55e5b4c797b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 >> 172.21.15.49:0/1218680464 conn(0x55e5b4c66f60 msgr2=0x55e5b4c68ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 shutdown_connections 2024-03-16T08:27:22.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 1 -- 172.21.15.49:0/1218680464 wait complete. 2024-03-16T08:27:22.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d45e9740 20 client.27884 trim_cache size 0 max 16384 2024-03-16T08:27:22.265 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:27:22.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 msgr2=0x7f55b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 0x7f55b40a4e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02c700 tx=0x7f55bc042000 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 msgr2=0x7f55cc198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e9f0 tx=0x7f55c000eeb0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 shutdown_connections 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f55b40a2970 0x7f55b40a4e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55cc107760 0x7f55cc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc069380 0x7f55cc198550 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 --2- 172.21.15.49:0/1336989290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc0689c0 0x7f55cc198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.256+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 >> 172.21.15.49:0/1336989290 conn(0x7f55cc0777e0 msgr2=0x7f55cc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.257+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 shutdown_connections 2024-03-16T08:27:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.257+0000 7f55d19f4640 1 -- 172.21.15.49:0/1336989290 wait complete. 2024-03-16T08:27:22.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T08:27:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:22 smithi129 ceph-mon[36561]: pgmap v315: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:22.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:22 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1612779985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.377+0000 7f0c37d60640 1 -- 172.21.15.49:0/2877561942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c30103370 msgr2=0x7f0c3010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.377+0000 7f0c37d60640 1 --2- 172.21.15.49:0/2877561942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c30103370 0x7f0c3010b850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009860 tx=0x7f0c2002bc00 comp rx=0 tx=0).stop 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.377+0000 7f0c37d60640 1 -- 172.21.15.49:0/2877561942 shutdown_connections 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.377+0000 7f0c37d60640 1 --2- 172.21.15.49:0/2877561942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3010be20 0x7f0c3010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 --2- 172.21.15.49:0/2877561942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c30103370 0x7f0c3010b850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 --2- 172.21.15.49:0/2877561942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30102a50 0x7f0c30102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- 172.21.15.49:0/2877561942 >> 172.21.15.49:0/2877561942 conn(0x7f0c30077840 msgr2=0x7f0c30077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- 172.21.15.49:0/2877561942 shutdown_connections 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- 172.21.15.49:0/2877561942 wait complete. 2024-03-16T08:27:22.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 Processor -- start 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- start start 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 0x7f0c301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 0x7f0c3019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111bf0 con 0x7f0c30102a50 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111a70 con 0x7f0c3010be20 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c37d60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111d70 con 0x7f0c30103370 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.378+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c362fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 0x7f0c3019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37484/0 (socket says 172.21.15.49:37484) 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c352fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 0x7f0c301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 -- 172.21.15.49:0/1712534295 learned_addr learned my addr 172.21.15.49:0/1712534295 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 -- 172.21.15.49:0/1712534295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 msgr2=0x7f0c301986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 0x7f0c301986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 -- 172.21.15.49:0/1712534295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 msgr2=0x7f0c3019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 0x7f0c3019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 -- 172.21.15.49:0/1712534295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3019d150 con 0x7f0c30102a50 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c352fb640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 0x7f0c301986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c362fd640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 0x7f0c3019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c35afc640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e9f0 tx=0x7f0c2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c26ffd640 1 -- 172.21.15.49:0/1712534295 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00cd80 con 0x7f0c30102a50 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3019d440 con 0x7f0c30102a50 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c26ffd640 1 -- 172.21.15.49:0/1712534295 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c005500 con 0x7f0c30102a50 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c26ffd640 1 -- 172.21.15.49:0/1712534295 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c010690 con 0x7f0c30102a50 2024-03-16T08:27:22.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.379+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a4cd0 con 0x7f0c30102a50 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.381+0000 7f0c26ffd640 1 -- 172.21.15.49:0/1712534295 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0c2c0046c0 con 0x7f0c30102a50 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.382+0000 7f0c3aa1f740 1 Processor -- start 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.382+0000 7f0c3aa1f740 1 -- start start 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9440ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c26ffd640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 0x7f0c1c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f944063490 con 0x55f9440ffe80 2024-03-16T08:27:22.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f94400bc90 con 0x55f94410d140 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f94405f530 con 0x55f9440ff560 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c352fb640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 0x7f0c1c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37500/0 (socket says 172.21.15.49:37500) 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c362fd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9440ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 -- 172.21.15.49:0/49649636 learned_addr learned my addr 172.21.15.49:0/49649636 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c26ffd640 1 -- 172.21.15.49:0/1712534295 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f0c2c0cf230 con 0x7f0c30102a50 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c362fd640 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 -- 172.21.15.49:0/49649636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 msgr2=0x55f9440ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9440ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 -- 172.21.15.49:0/49649636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 msgr2=0x55f94410f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 -- 172.21.15.49:0/49649636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9440c6500 con 0x55f9440ffe80 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c362fd640 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9440ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c352fb640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 0x7f0c1c0a4e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002c210 tx=0x7f0c2002e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c35afc640 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c09b840 tx=0x7f0c2c01bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adff46ea6d173ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.383+0000 7f0c1bfff640 1 -- 172.21.15.49:0/49649636 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c09b210 con 0x55f9440ffe80 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c1bfff640 1 -- 172.21.15.49:0/49649636 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c09bfb0 con 0x55f9440ffe80 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c1bfff640 1 -- 172.21.15.49:0/49649636 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c097f20 con 0x55f9440ffe80 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/49649636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 msgr2=0x55f94410cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c09b840 tx=0x7f0c2c01bb30 comp rx=0 tx=0).stop 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/49649636 shutdown_connections 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94410cc00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/49649636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9440ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/49649636 >> 172.21.15.49:0/49649636 conn(0x55f9440f9f60 msgr2=0x55f9440fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/49649636 shutdown_connections 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.384+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/49649636 wait complete. 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 -- start start 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 Processor -- start 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ffe80 0x55f94410a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f94410ff00 con 0x55f9440ff560 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f94401f230 con 0x55f94410d140 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f944110080 con 0x55f9440ffe80 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c3aa1f740 10 client.0 _ll_register_callbacks cb 0x55f94405b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c35afc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37508/0 (socket says 172.21.15.49:37508) 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c35afc640 1 -- 172.21.15.49:0/986186617 learned_addr learned my addr 172.21.15.49:0/986186617 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c362fd640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.385+0000 7f0c352fb640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ffe80 0x55f94410a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 -- 172.21.15.49:0/986186617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ffe80 msgr2=0x55f94410a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ffe80 0x55f94410a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 -- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 msgr2=0x55f94410abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 -- 172.21.15.49:0/986186617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94402e100 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c362fd640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c35afc640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c098280 tx=0x7f0c2c098150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b786ac30f70da29b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c09b210 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f0c2c099950 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2c09bfb0 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c2c09fa00 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c3aa1f740 10 client.18015 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f943d773f0 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c09bfb0 con 0x55f9440ff560 2024-03-16T08:27:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.386+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0c2c0a3bc0 con 0x55f9440ff560 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 10 client.18015 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 10 client.18015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 20 client.18015 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f94411d310 0x55f94411f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 4 client.18015 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9440709d0 con 0x55f94411d310 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c352fb640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f94411d310 0x55f94411f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c197fa640 10 client.18015 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.387+0000 7f0c352fb640 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f94411d310 0x55f94411f700 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc78cf5ecd46933 server_cookie=10cbda19ab886ac0 in_seq=0 out_seq=0 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c197fa640 1 -- 172.21.15.49:0/986186617 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f9440709d0 con 0x55f94411d310 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c197fa640 10 client.18015 handle_command_reply: tid=0 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 client.18015 shutdown 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 msgr2=0x55f94410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c098280 tx=0x7f0c2c098150 comp rx=0 tx=0).stop 2024-03-16T08:27:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 shutdown_connections 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f94411d310 0x55f94411f700 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f94410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ffe80 0x55f94410a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/986186617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ff560 0x55f94410a130 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c197fa640 0 client.18015 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 >> 172.21.15.49:0/986186617 conn(0x55f9440f9f60 msgr2=0x55f9440fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 shutdown_connections 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/986186617 wait complete. 2024-03-16T08:27:22.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.388+0000 7f0c3aa1f740 20 client.18015 trim_cache size 0 max 16384 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 Processor -- start 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 -- start start 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9441098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f9441078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301adb30 con 0x55f9440ffe80 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ab400 con 0x55f94410d140 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.401+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301a9250 con 0x55f9440ff560 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37524/0 (socket says 172.21.15.49:37524) 2024-03-16T08:27:22.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 -- 172.21.15.49:0/4089324967 learned_addr learned my addr 172.21.15.49:0/4089324967 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c362fd640 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f9441078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c35afc640 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9441098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 -- 172.21.15.49:0/4089324967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 msgr2=0x55f9441098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9441098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 -- 172.21.15.49:0/4089324967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 msgr2=0x55f9441078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f9441078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 -- 172.21.15.49:0/4089324967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9440ffcb0 con 0x55f9440ffe80 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c352fb640 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002c140 tx=0x7f0c2003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c659cd7e4a203c5e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c1b7fe640 1 -- 172.21.15.49:0/4089324967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20039040 con 0x55f9440ffe80 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c1b7fe640 1 -- 172.21.15.49:0/4089324967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20059070 con 0x55f9440ffe80 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.402+0000 7f0c1b7fe640 1 -- 172.21.15.49:0/4089324967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2002c800 con 0x55f9440ffe80 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/4089324967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 msgr2=0x55f944107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002c140 tx=0x7f0c2003e000 comp rx=0 tx=0).stop 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/4089324967 shutdown_connections 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f94410d140 0x55f9441078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f944107390 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/4089324967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9440ff560 0x55f9441098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/4089324967 >> 172.21.15.49:0/4089324967 conn(0x55f9440f9f60 msgr2=0x55f9440f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/4089324967 shutdown_connections 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/4089324967 wait complete. 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 -- start start 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.403+0000 7f0c3aa1f740 1 Processor -- start 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9440ff560 0x55f94411f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f94410d140 0x55f94411fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301a98c0 con 0x55f9440ffe80 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301aad30 con 0x55f9440ff560 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301a9f90 con 0x55f94410d140 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 10 client.0 _ll_register_callbacks cb 0x7f0c2c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37534/0 (socket says 172.21.15.49:37534) 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 -- 172.21.15.49:0/2492248113 learned_addr learned my addr 172.21.15.49:0/2492248113 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c35afc640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9440ff560 0x55f94411f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c362fd640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f94410d140 0x55f94411fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 -- 172.21.15.49:0/2492248113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f94410d140 msgr2=0x55f94411fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f94410d140 0x55f94411fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 -- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9440ff560 msgr2=0x55f94411f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9440ff560 0x55f94411f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 -- 172.21.15.49:0/2492248113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f943e417d0 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c352fb640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0c20037040 tx=0x7f0c20005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1429c533b3fcf35e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20061020 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f0c20054310 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.404+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20059070 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c20039040 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 10 client.18021 fetch_fsmap learned FSMap version 24 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f944182570 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20059070 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0c2004a050 con 0x55f9440ffe80 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 10 client.18021 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T08:27:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 20 client.18021 populate_metadata read hostname 'smithi049' 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f944182d70 0x55f944185190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 4 client.18021 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 --> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f94406ad20 con 0x55f944182d70 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.405+0000 7f0c35afc640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f944182d70 0x55f944185190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c197fa640 10 client.18021 ms_handle_connect on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c35afc640 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f944182d70 0x55f944185190 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d6f5fbd206258b server_cookie=f05a29289d2367e2 in_seq=0 out_seq=0 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c197fa640 1 -- 172.21.15.49:0/2492248113 <== mds.0 v2:172.21.15.129:6838/1288779286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f94406ad20 con 0x55f944182d70 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c197fa640 10 client.18021 handle_command_reply: tid=0 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 client.18021 shutdown 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 msgr2=0x55f94411f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0c20037040 tx=0x7f0c20005fa0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 shutdown_connections 2024-03-16T08:27:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:6838/1288779286,v1:172.21.15.129:6839/1288779286] conn(0x55f944182d70 0x55f944185190 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f94410d140 0x55f94411fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f9440ffe80 0x55f94411f700 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c197fa640 0 client.18021 ms_handle_reset on v2:172.21.15.129:6838/1288779286 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 --2- 172.21.15.49:0/2492248113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9440ff560 0x55f94411f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 >> 172.21.15.49:0/2492248113 conn(0x55f9440f9f60 msgr2=0x55f9440f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 shutdown_connections 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 1 -- 172.21.15.49:0/2492248113 wait complete. 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.406+0000 7f0c3aa1f740 20 client.18021 trim_cache size 0 max 16384 2024-03-16T08:27:22.415 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-03-16T08:27:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 msgr2=0x7f0c1c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 0x7f0c1c0a4e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2002c210 tx=0x7f0c2002e040 comp rx=0 tx=0).stop 2024-03-16T08:27:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 msgr2=0x7f0c301981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e9f0 tx=0x7f0c2c00edf0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 shutdown_connections 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f0c1c0a2970 0x7f0c1c0a4e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3010be20 0x7f0c3019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c30103370 0x7f0c301986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 --2- 172.21.15.49:0/1712534295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c30102a50 0x7f0c301981b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 >> 172.21.15.49:0/1712534295 conn(0x7f0c30077840 msgr2=0x7f0c300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 shutdown_connections 2024-03-16T08:27:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:22.407+0000 7f0c37d60640 1 -- 172.21.15.49:0/1712534295 wait complete. 2024-03-16T08:27:22.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:22 smithi154 ceph-mon[36471]: pgmap v315: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:22.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:22 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1612779985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:27:23.420 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-16T08:27:23.420 INFO:tasks.fwd_scrub:done joining 2024-03-16T08:27:23.420 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-16T08:27:23.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2958752710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0102130 msgr2=0x7fd6e010e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2958752710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0102130 0x7fd6e010e640 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009cd0 tx=0x7fd6d802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2958752710 shutdown_connections 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2958752710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0102130 0x7fd6e010e640 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2958752710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0101770 0x7fd6e0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2958752710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e0107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2958752710 >> 172.21.15.49:0/2958752710 conn(0x7fd6e00fd280 msgr2=0x7fd6e00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2958752710 shutdown_connections 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.554+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2958752710 wait complete. 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 Processor -- start 2024-03-16T08:27:23.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 -- start start 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0102130 0x7fd6e01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111b90 con 0x7fd6e0101770 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111a10 con 0x7fd6e0107780 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e79cd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111d10 con 0x7fd6e0102130 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e5769640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e5769640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43516/0 (socket says 172.21.15.49:43516) 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e5769640 1 -- 172.21.15.49:0/2360647647 learned_addr learned my addr 172.21.15.49:0/2360647647 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e4f68640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0102130 0x7fd6e01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.555+0000 7fd6e5f6a640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 -- 172.21.15.49:0/2360647647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0102130 msgr2=0x7fd6e01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0102130 0x7fd6e01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 -- 172.21.15.49:0/2360647647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 msgr2=0x7fd6e019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 -- 172.21.15.49:0/2360647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e019d060 con 0x7fd6e0101770 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5f6a640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e5769640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e9f0 tx=0x7fd6dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00cd80 con 0x7fd6e0101770 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc010910 con 0x7fd6e0101770 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e019d350 con 0x7fd6e0101770 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc018990 con 0x7fd6e0101770 2024-03-16T08:27:23.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.556+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a4be0 con 0x7fd6e0101770 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.557+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e0068a20 con 0x7fd6e0101770 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.558+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd6dc010ac0 con 0x7fd6e0101770 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.559+0000 7fd6d67fc640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 0x7fd6cc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.559+0000 7fd6e4f68640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 0x7fd6cc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.559+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7fd6dc0cf1b0 con 0x7fd6e0101770 2024-03-16T08:27:23.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.559+0000 7fd6e4f68640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 0x7fd6cc0a4e30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ff8d0 tx=0x7fd6d0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:23.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.563+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd6dc09bf30 con 0x7fd6e0101770 2024-03-16T08:27:23.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.749+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6e01993c0 con 0x7fd6e0101770 2024-03-16T08:27:23.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.751+0000 7fd6d67fc640 1 -- 172.21.15.49:0/2360647647 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23055 (secure 0 0 0) 0x7fd6dc09b830 con 0x7fd6e0101770 2024-03-16T08:27:23.752 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:27:23.752 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":97,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:27:14.055824+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/1561370229":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3786569188":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3762364594":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/236450725":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 msgr2=0x7fd6cc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 0x7fd6cc0a4e30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ff8d0 tx=0x7fd6d0005f90 comp rx=0 tx=0).stop 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 msgr2=0x7fd6e0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e9f0 tx=0x7fd6dc00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 shutdown_connections 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6cc0a2970 0x7fd6cc0a4e30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0107780 0x7fd6e019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0102130 0x7fd6e01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 --2- 172.21.15.49:0/2360647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6e0101770 0x7fd6e0198070 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 >> 172.21.15.49:0/2360647647 conn(0x7fd6e00fd280 msgr2=0x7fd6e00fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 shutdown_connections 2024-03-16T08:27:23.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.753+0000 7fd6e79cd640 1 -- 172.21.15.49:0/2360647647 wait complete. 2024-03-16T08:27:23.765 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:27:23.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.876+0000 7f84ac17e640 1 -- 172.21.15.49:0/3867890571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a4103370 msgr2=0x7f84a410b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.876+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3867890571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a4103370 0x7f84a410b850 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8498009900 tx=0x7f849802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- 172.21.15.49:0/3867890571 shutdown_connections 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3867890571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a410be20 0x7f84a410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3867890571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a4103370 0x7f84a410b850 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3867890571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 0x7f84a4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- 172.21.15.49:0/3867890571 >> 172.21.15.49:0/3867890571 conn(0x7f84a4077840 msgr2=0x7f84a4077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- 172.21.15.49:0/3867890571 shutdown_connections 2024-03-16T08:27:23.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- 172.21.15.49:0/3867890571 wait complete. 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 Processor -- start 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- start start 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 0x7f84a4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a4103370 0x7f84a4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4111bf0 con 0x7f84a410be20 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4111a70 con 0x7f84a4102a50 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.877+0000 7f84ac17e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4111d70 con 0x7f84a4103370 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84a9719640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a4103370 0x7f84a4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84a9f1a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 0x7f84a4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43534/0 (socket says 172.21.15.49:43534) 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 -- 172.21.15.49:0/3873904806 learned_addr learned my addr 172.21.15.49:0/3873904806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 -- 172.21.15.49:0/3873904806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a4103370 msgr2=0x7f84a4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a4103370 0x7f84a4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 -- 172.21.15.49:0/3873904806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 msgr2=0x7f84a4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 0x7f84a4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 -- 172.21.15.49:0/3873904806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419d100 con 0x7f84a410be20 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84aa71b640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9b0 tx=0x7f849c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c00cd80 con 0x7f84a410be20 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419d3f0 con 0x7f84a410be20 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849c0035f0 con 0x7f84a410be20 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c010670 con 0x7f84a410be20 2024-03-16T08:27:23.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.878+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a4c80 con 0x7f84a410be20 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.879+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4068a20 con 0x7f84a410be20 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.880+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f849c0037a0 con 0x7f84a410be20 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.881+0000 7f84a2ffd640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 0x7f848c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.881+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f849c0cf0b0 con 0x7f84a410be20 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.881+0000 7f84a9f1a640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 0x7f848c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:23.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.882+0000 7f84a9f1a640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 0x7f848c0a4e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f8494009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:23.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:23.884+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f849c09bf60 con 0x7f84a410be20 2024-03-16T08:27:24.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.066+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84a4106f70 con 0x7f84a410be20 2024-03-16T08:27:24.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.067+0000 7f84a2ffd640 1 -- 172.21.15.49:0/3873904806 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f849c09b860 con 0x7f84a410be20 2024-03-16T08:27:24.067 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 msgr2=0x7f848c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 0x7f848c0a4e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f8494009500 comp rx=0 tx=0).stop 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 msgr2=0x7f84a419c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9b0 tx=0x7f849c00ee70 comp rx=0 tx=0).stop 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 shutdown_connections 2024-03-16T08:27:24.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f848c0a2970 0x7f848c0a4e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84a410be20 0x7f84a419c9d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84a4103370 0x7f84a4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 --2- 172.21.15.49:0/3873904806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84a4102a50 0x7f84a4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 >> 172.21.15.49:0/3873904806 conn(0x7f84a4077840 msgr2=0x7f84a40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 shutdown_connections 2024-03-16T08:27:24.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.069+0000 7f84ac17e640 1 -- 172.21.15.49:0/3873904806 wait complete. 2024-03-16T08:27:24.080 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 -- 172.21.15.49:0/3969352472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c102a50 msgr2=0x7f427c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 --2- 172.21.15.49:0/3969352472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c102a50 0x7f427c102e30 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4278009900 tx=0x7f427802bcd0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 -- 172.21.15.49:0/3969352472 shutdown_connections 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 --2- 172.21.15.49:0/3969352472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c10be20 0x7f427c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 --2- 172.21.15.49:0/3969352472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 0x7f427c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 --2- 172.21.15.49:0/3969352472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c102a50 0x7f427c102e30 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 -- 172.21.15.49:0/3969352472 >> 172.21.15.49:0/3969352472 conn(0x7f427c077840 msgr2=0x7f427c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 -- 172.21.15.49:0/3969352472 shutdown_connections 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.191+0000 7f4283426640 1 -- 172.21.15.49:0/3969352472 wait complete. 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 Processor -- start 2024-03-16T08:27:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 -- start start 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c102a50 0x7f427c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 0x7f427c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f427c111bf0 con 0x7f427c10be20 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f427c111a70 con 0x7f427c102a50 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f4283426640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f427c111d70 con 0x7f427c103370 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f42819c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f42819c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43554/0 (socket says 172.21.15.49:43554) 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f42819c3640 1 -- 172.21.15.49:0/631146559 learned_addr learned my addr 172.21.15.49:0/631146559 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f42811c2640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c102a50 0x7f427c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.192+0000 7f42809c1640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 0x7f427c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 -- 172.21.15.49:0/631146559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 msgr2=0x7f427c100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 0x7f427c100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 -- 172.21.15.49:0/631146559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c102a50 msgr2=0x7f427c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c102a50 0x7f427c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 -- 172.21.15.49:0/631146559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c1a6590 con 0x7f427c10be20 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42819c3640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f427400efd0 tx=0x7f427400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274003b70 con 0x7f427c10be20 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4274003d10 con 0x7f427c10be20 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c1a6880 con 0x7f427c10be20 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740109f0 con 0x7f427c10be20 2024-03-16T08:27:24.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.193+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1a6d80 con 0x7f427c10be20 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.194+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c068a20 con 0x7f427c10be20 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.195+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4274010b90 con 0x7f427c10be20 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.196+0000 7f42727fc640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 0x7f42680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.196+0000 7f42811c2640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 0x7f42680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.196+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f42740cf5b0 con 0x7f427c10be20 2024-03-16T08:27:24.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.196+0000 7f42811c2640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 0x7f42680a4e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f427802efd0 tx=0x7f427802cf90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.199+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f427409c330 con 0x7f427c10be20 2024-03-16T08:27:24.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:24 smithi129 ceph-mon[36561]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:24 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2360647647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:27:24.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:24 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3873904806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:27:24.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.382+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f427c106f10 con 0x7f427c10be20 2024-03-16T08:27:24.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.383+0000 7f42727fc640 1 -- 172.21.15.49:0/631146559 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f427409bc30 con 0x7f427c10be20 2024-03-16T08:27:24.383 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 msgr2=0x7f42680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 0x7f42680a4e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f427802efd0 tx=0x7f427802cf90 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 msgr2=0x7f427c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f427400efd0 tx=0x7f427400c6a0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 shutdown_connections 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f42680a2970 0x7f42680a4e30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c10be20 0x7f427c101120 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f427c103370 0x7f427c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 --2- 172.21.15.49:0/631146559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f427c102a50 0x7f427c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 >> 172.21.15.49:0/631146559 conn(0x7f427c077840 msgr2=0x7f427c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 shutdown_connections 2024-03-16T08:27:24.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.385+0000 7f4283426640 1 -- 172.21.15.49:0/631146559 wait complete. 2024-03-16T08:27:24.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:24 smithi049 ceph-mon[31942]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:24.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:24 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2360647647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:27:24.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:24 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3873904806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:27:24.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:27:24.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:24 smithi154 ceph-mon[36471]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-16T08:27:24.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:24 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2360647647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:27:24.476 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:24 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3873904806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 -- 172.21.15.49:0/3519602010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 msgr2=0x7f4530100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 --2- 172.21.15.49:0/3519602010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f4530100b80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4520009930 tx=0x7f452002bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 -- 172.21.15.49:0/3519602010 shutdown_connections 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 --2- 172.21.15.49:0/3519602010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 0x7f453010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 --2- 172.21.15.49:0/3519602010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f4530100b80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 --2- 172.21.15.49:0/3519602010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 0x7f4530106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 -- 172.21.15.49:0/3519602010 >> 172.21.15.49:0/3519602010 conn(0x7f4530077840 msgr2=0x7f4530077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 -- 172.21.15.49:0/3519602010 shutdown_connections 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 -- 172.21.15.49:0/3519602010 wait complete. 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.507+0000 7f4537d74640 1 Processor -- start 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 -- start start 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 0x7f4530198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 0x7f453019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110b20 con 0x7f4530100700 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45301109a0 con 0x7f4530106710 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110ca0 con 0x7f45301010c0 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43570/0 (socket says 172.21.15.49:43570) 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 -- 172.21.15.49:0/923969287 learned_addr learned my addr 172.21.15.49:0/923969287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4536311640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 0x7f453019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f453530f640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 0x7f4530198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 -- 172.21.15.49:0/923969287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 msgr2=0x7f4530198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 0x7f4530198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 -- 172.21.15.49:0/923969287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 msgr2=0x7f453019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 0x7f453019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 -- 172.21.15.49:0/923969287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453019d1b0 con 0x7f4530100700 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4535b10640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f452c00e9f0 tx=0x7f452c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c00cd80 con 0x7f4530100700 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452c010910 con 0x7f4530100700 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453019d4a0 con 0x7f4530100700 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.508+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c018990 con 0x7f4530100700 2024-03-16T08:27:24.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.509+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301a4d30 con 0x7f4530100700 2024-03-16T08:27:24.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.510+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4530068a20 con 0x7f4530100700 2024-03-16T08:27:24.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.511+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f452c010ac0 con 0x7f4530100700 2024-03-16T08:27:24.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.511+0000 7f4526ffd640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 0x7f451c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.511+0000 7f453530f640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 0x7f451c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.511+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f452c0cf050 con 0x7f4530100700 2024-03-16T08:27:24.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.512+0000 7f453530f640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 0x7f451c0a4eb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4520005ab0 tx=0x7f4520005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.516+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f452c09b780 con 0x7f4530100700 2024-03-16T08:27:24.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.698+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4530109c80 con 0x7f4530100700 2024-03-16T08:27:24.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.698+0000 7f4526ffd640 1 -- 172.21.15.49:0/923969287 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f452c09f820 con 0x7f4530100700 2024-03-16T08:27:24.699 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 msgr2=0x7f451c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 0x7f451c0a4eb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4520005ab0 tx=0x7f4520005fd0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 msgr2=0x7f45301981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f452c00e9f0 tx=0x7f452c00eeb0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 shutdown_connections 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f451c0a29f0 0x7f451c0a4eb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4530106710 0x7f453019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45301010c0 0x7f4530198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 --2- 172.21.15.49:0/923969287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4530100700 0x7f45301981c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.700+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 >> 172.21.15.49:0/923969287 conn(0x7f4530077840 msgr2=0x7f453006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.701+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 shutdown_connections 2024-03-16T08:27:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.701+0000 7f4537d74640 1 -- 172.21.15.49:0/923969287 wait complete. 2024-03-16T08:27:24.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- 172.21.15.49:0/1783207032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 msgr2=0x7f051010d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1783207032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f051010d5b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009930 tx=0x7f04f802bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- 172.21.15.49:0/1783207032 shutdown_connections 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1783207032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f051010d5b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1783207032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1783207032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f0510106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- 172.21.15.49:0/1783207032 >> 172.21.15.49:0/1783207032 conn(0x7f05100777e0 msgr2=0x7f0510077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- 172.21.15.49:0/1783207032 shutdown_connections 2024-03-16T08:27:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- 172.21.15.49:0/1783207032 wait complete. 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 Processor -- start 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 -- start start 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.820+0000 7f0516fd4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f0510199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f05101a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0510110ac0 con 0x7f05101066f0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0510110940 con 0x7f05101006e0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0510110c40 con 0x7f05101010a0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0514d70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f05101a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0514d70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.49:36156/0 (socket says 172.21.15.49:36156) 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0514d70640 1 -- 172.21.15.49:0/1477453640 learned_addr learned my addr 172.21.15.49:0/1477453640 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f050ffff640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f0510199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 -- 172.21.15.49:0/1477453640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 msgr2=0x7f0510199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f0510199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 -- 172.21.15.49:0/1477453640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 msgr2=0x7f0510198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 -- 172.21.15.49:0/1477453640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05101a44c0 con 0x7f05101066f0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0514d70640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0515571640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f05101a3f80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f04f802c170 tx=0x7f04f802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8041070 con 0x7f05101066f0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05101a4690 con 0x7f05101066f0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f8004450 con 0x7f05101066f0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.821+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8003d00 con 0x7f05101066f0 2024-03-16T08:27:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.822+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a4aa0 con 0x7f05101066f0 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.823+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05100685d0 con 0x7f05101066f0 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.824+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f04f8044070 con 0x7f05101066f0 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.824+0000 7f050dffb640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 0x7f04fc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.825+0000 7f0514d70640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 0x7f04fc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.825+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f04f80f0910 con 0x7f05101066f0 2024-03-16T08:27:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.825+0000 7f0514d70640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 0x7f04fc0a4e30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05080117b0 tx=0x7f0508009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:24.828+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04f80bd150 con 0x7f05101066f0 2024-03-16T08:27:25.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.062+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f05101086e0 con 0x7f05101066f0 2024-03-16T08:27:25.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.071+0000 7f050dffb640 1 -- 172.21.15.49:0/1477453640 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740361 (secure 0 0 0) 0x7f04f80c11f0 con 0x7f05101066f0 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 25, 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "active_gid": 15432, 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "active_name": "x", 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addrs": { 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-03-16T08:27:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4265921640 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4265921640 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addr": "172.21.15.49:6801/4265921640", 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "active_change": "2024-03-16T08:17:12.732463+0000", 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "active_mgr_features": 4540701547738038271, 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "available": true, 2024-03-16T08:27:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "standbys": [ 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 15420, 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "y", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-16T08:27:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T08:27:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-16T08:27:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-16T08:27:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-16T08:27:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-16T08:27:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-16T08:27:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-16T08:27:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-16T08:27:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-16T08:27:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T08:27:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-16T08:27:25.123 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.127 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-16T08:27:25.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-16T08:27:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.133 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.134 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.135 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-16T08:27:25.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-16T08:27:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-16T08:27:25.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-16T08:27:25.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-16T08:27:25.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-16T08:27:25.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-16T08:27:25.142 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T08:27:25.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.146 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.150 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-16T08:27:25.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.153 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.157 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.158 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-16T08:27:25.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-16T08:27:25.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-16T08:27:25.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-16T08:27:25.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-16T08:27:25.165 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-16T08:27:25.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-16T08:27:25.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.168 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-16T08:27:25.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.170 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-16T08:27:25.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-16T08:27:25.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.176 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.178 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.180 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.182 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-16T08:27:25.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T08:27:25.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T08:27:25.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T08:27:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.190 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.196 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.198 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.201 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.202 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.204 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.206 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.210 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.213 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.214 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T08:27:25.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T08:27:25.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T08:27:25.217 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.218 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T08:27:25.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T08:27:25.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T08:27:25.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.225 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.227 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.229 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.231 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.232 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:25.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.234 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.235 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.236 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.237 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.240 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.242 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-16T08:27:25.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.244 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:25.246 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.247 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:25.248 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.250 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-16T08:27:25.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.254 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.255 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.256 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.257 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.259 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-16T08:27:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.261 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.262 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.264 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.266 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.269 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.270 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.272 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-16T08:27:25.275 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.278 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-16T08:27:25.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.281 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.284 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-16T08:27:25.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.293 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.296 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.297 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.298 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.299 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.300 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.302 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.303 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.304 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.305 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T08:27:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.307 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.308 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.310 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.311 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.312 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-16T08:27:25.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.315 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.316 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.317 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.318 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.319 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-16T08:27:25.321 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.322 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.323 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.324 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-16T08:27:25.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.330 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.332 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.338 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.340 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.342 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.343 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.345 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.347 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.348 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.350 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:25.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.354 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.355 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.356 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.358 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.364 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.365 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.369 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.370 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.371 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.377 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.378 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.379 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.380 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.382 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.390 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.393 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-16T08:27:25.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.410 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.414 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.416 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.419 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.420 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.426 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.430 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.432 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.434 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.438 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.439 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.441 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-16T08:27:25.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-16T08:27:25.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-16T08:27:25.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.446 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.447 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.448 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.449 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.451 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.456 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.461 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.462 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.465 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.467 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.468 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.469 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.470 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-16T08:27:25.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-16T08:27:25.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-16T08:27:25.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-16T08:27:25.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:25.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.492 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-16T08:27:25.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.494 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.495 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.496 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.497 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.506 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.511 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.512 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T08:27:25.517 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.519 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.521 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.523 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.524 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.526 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.528 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-16T08:27:25.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.531 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.532 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25316, 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "z", 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.534 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.539 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-16T08:27:25.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-16T08:27:25.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-16T08:27:25.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-16T08:27:25.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-16T08:27:25.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-16T08:27:25.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.556 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.557 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:25.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.560 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.563 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.564 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:25.566 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.568 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.571 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.572 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.580 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.588 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.589 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T08:27:25.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-16T08:27:25.592 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-16T08:27:25.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T08:27:25.599 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-16T08:27:25.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-16T08:27:25.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.612 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T08:27:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-16T08:27:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-16T08:27:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-16T08:27:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-16T08:27:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-16T08:27:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-16T08:27:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-16T08:27:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T08:27:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T08:27:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T08:27:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T08:27:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T08:27:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-16T08:27:25.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-16T08:27:25.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.712 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-16T08:27:25.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.718 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-16T08:27:25.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-16T08:27:25.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.727 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.730 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-16T08:27:25.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-16T08:27:25.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T08:27:25.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.744 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-16T08:27:25.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T08:27:25.754 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.755 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.762 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-16T08:27:25.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-16T08:27:25.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:25.767 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.769 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.770 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.771 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.772 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.772 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.773 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.776 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.778 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.779 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.782 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.783 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-16T08:27:25.784 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.787 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.789 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.790 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.791 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.793 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.794 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.795 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.798 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-16T08:27:25.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-16T08:27:25.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.805 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.806 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.807 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.810 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.811 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.812 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.813 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.815 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.816 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.818 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.822 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.825 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.826 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.829 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.831 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-16T08:27:25.832 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.833 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.834 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.836 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.837 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.838 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.839 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.840 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.841 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.843 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-16T08:27:25.845 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.846 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.849 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:25.853 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-16T08:27:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T08:27:25.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-16T08:27:25.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:25.863 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.866 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.872 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.875 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.877 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.878 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.883 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.884 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.886 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.887 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.889 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.890 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.893 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.895 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.896 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.898 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.900 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.902 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.903 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.904 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.906 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.909 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.910 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.912 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.913 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.914 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.915 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.917 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.920 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.923 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.924 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.925 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:25.927 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-16T08:27:25.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.929 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.930 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.931 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-16T08:27:25.932 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.934 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.937 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.938 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.940 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.941 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.942 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.943 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.945 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.946 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.949 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.950 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.951 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.953 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.954 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.956 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.957 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.958 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.960 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.962 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.963 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.965 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.966 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-16T08:27:25.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T08:27:25.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.971 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.972 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-16T08:27:25.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.974 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.976 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-16T08:27:25.978 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-16T08:27:25.981 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.982 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.983 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.986 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.989 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.990 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.992 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.993 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.994 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:25.996 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:25.997 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:25.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:25.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.000 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.001 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.002 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.003 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.006 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-16T08:27:26.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.009 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.011 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-16T08:27:26.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.013 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.014 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.015 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.017 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.018 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.019 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-16T08:27:26.021 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.022 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "modules": [ 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard", 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "iostat", 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "nfs", 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "restful", 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_schedule" 2024-03-16T08:27:26.024 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.025 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.026 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.027 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.028 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.030 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.031 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.032 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-16T08:27:26.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-16T08:27:26.035 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.037 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-16T08:27:26.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.039 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.040 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.041 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T08:27:26.043 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-16T08:27:26.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-16T08:27:26.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-16T08:27:26.048 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T08:27:26.050 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.051 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.052 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.053 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.054 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-16T08:27:26.058 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.059 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.060 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-16T08:27:26.061 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.062 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.064 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-16T08:27:26.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T08:27:26.068 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.069 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-16T08:27:26.070 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.071 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T08:27:26.073 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.074 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.075 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.076 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-16T08:27:26.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.079 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.080 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-16T08:27:26.081 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.082 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T08:27:26.083 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-16T08:27:26.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.086 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.088 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.089 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T08:27:26.091 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-16T08:27:26.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.094 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.098 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-16T08:27:26.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-16T08:27:26.101 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.105 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-16T08:27:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.109 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-16T08:27:26.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-16T08:27:26.117 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.118 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-16T08:27:26.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.123 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.124 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-16T08:27:26.126 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-16T08:27:26.130 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.132 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.133 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.134 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.135 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.136 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.139 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-16T08:27:26.146 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.149 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T08:27:26.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.158 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.160 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.161 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T08:27:26.165 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.166 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.171 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T08:27:26.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T08:27:26.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.185 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T08:27:26.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T08:27:26.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.192 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.193 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.195 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.197 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.198 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.199 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.201 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.202 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.203 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.205 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:26.206 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.209 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.210 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.212 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.217 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.219 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.222 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.223 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-16T08:27:26.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-16T08:27:26.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.228 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.229 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.231 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.232 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.233 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.235 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-16T08:27:26.237 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.238 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-16T08:27:26.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.247 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.249 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.250 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-16T08:27:26.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.252 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-16T08:27:26.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.258 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.262 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.265 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.266 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.270 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.271 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.272 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.274 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.276 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.277 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.279 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.281 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.282 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.284 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.286 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T08:27:26.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.291 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.293 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.294 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.295 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-16T08:27:26.296 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.297 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.299 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.300 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.301 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.302 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.303 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.304 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.306 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.307 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.308 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.310 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.312 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.313 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.315 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.318 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.319 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.320 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.321 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.322 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.323 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.327 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.328 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.329 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.330 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.331 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.332 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.333 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.334 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.338 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.339 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.341 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.342 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.344 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.345 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-16T08:27:26.346 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-16T08:27:26.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.349 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.350 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.352 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.353 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.356 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-16T08:27:26.364 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.365 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.367 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.371 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-16T08:27:26.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-16T08:27:26.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.381 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.382 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.384 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.386 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.389 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.390 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.393 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-16T08:27:26.395 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.396 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.398 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.399 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.401 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-16T08:27:26.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-16T08:27:26.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.416 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-16T08:27:26.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-16T08:27:26.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.425 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.426 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.428 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.429 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.435 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.441 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-16T08:27:26.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.445 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.447 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.448 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-16T08:27:26.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.455 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T08:27:26.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-16T08:27:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.465 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.468 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.469 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.471 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.473 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.474 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.478 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.483 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.484 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.487 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-16T08:27:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-16T08:27:26.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.492 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-16T08:27:26.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.498 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.503 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.504 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-16T08:27:26.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "services": { 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard": "https://172.21.15.49:8443/" 2024-03-16T08:27:26.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "always_on_modules": { 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "octopus": [ 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-16T08:27:26.507 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "pacific": [ 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "quincy": [ 2024-03-16T08:27:26.508 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "reef": [ 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-16T08:27:26.509 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "squid": [ 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-16T08:27:26.510 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "last_failure_osd_epoch": 89, 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: "active_clients": [ 2024-03-16T08:27:26.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1979333516 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "libcephsqlite", 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4186315294 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2112551280 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.514 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1178142679 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 35613494 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2336137393 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:26.516 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-16T08:27:26.517 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/631146559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/923969287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:25 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1477453640' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:26 smithi154 ceph-mon[36471]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/631146559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/923969287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:25 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1477453640' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T08:27:26.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:26 smithi129 ceph-mon[36561]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 msgr2=0x7f04fc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 0x7f04fc0a4e30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05080117b0 tx=0x7f0508009500 comp rx=0 tx=0).stop 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 msgr2=0x7f05101a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f05101a3f80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f04f802c170 tx=0x7f04f802cb30 comp rx=0 tx=0).stop 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 shutdown_connections 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f04fc0a2970 0x7f04fc0a4e30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05101066f0 0x7f05101a3f80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:26.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05101010a0 0x7f0510199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:26.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 --2- 172.21.15.49:0/1477453640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05101006e0 0x7f0510198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:26.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 >> 172.21.15.49:0/1477453640 conn(0x7f05100777e0 msgr2=0x7f0510102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:26.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 shutdown_connections 2024-03-16T08:27:26.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:25.076+0000 7f0516fd4640 1 -- 172.21.15.49:0/1477453640 wait complete. 2024-03-16T08:27:26.527 DEBUG:tasks.check_counter:Getting stats from a 2024-03-16T08:27:26.527 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-16T08:27:26.527 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/631146559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.528 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/923969287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:27:26.528 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:25 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1477453640' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T08:27:26.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:26 smithi049 ceph-mon[31942]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-16T08:27:26.799 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:27:27.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:27 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:27:27.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:27 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:27:27.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:27 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:27:28.220 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:28 smithi049 ceph-mon[31942]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T08:27:28.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:28 smithi129 ceph-mon[36561]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T08:27:28.475 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:28 smithi154 ceph-mon[36471]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T08:27:28.513 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:28.513 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:28.513 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 213, 2024-03-16T08:27:28.513 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 203, 2024-03-16T08:27:28.513 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1435184, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 124704, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.041713952, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.013691585, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.029800905, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.004251581, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 202, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.010174896, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000050370 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:28.514 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1435184, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 124704 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 6, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 157, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3344, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 183328, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:28.515 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.013596230, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.009064507, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002271339, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 155, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.014171315, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000091427 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:28.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 183328 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.000422216, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000353493, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:28.517 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:28.518 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:28.519 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:28.520 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148994, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 31, 2024-03-16T08:27:28.521 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-16T08:27:28.522 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 215, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1405179, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 215, 2024-03-16T08:27:28.523 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 215, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1405179, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:28.524 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:29.151 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-16T08:27:29.151 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-16T08:27:29.151 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-16T08:27:29.151 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-16T08:27:29.152 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-16T08:27:29.152 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-16T08:27:29.152 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-16T08:27:29.152 DEBUG:tasks.check_counter:Getting stats from d 2024-03-16T08:27:29.152 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:29 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:29 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.879 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:29.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:29 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:30 smithi129 ceph-mon[36561]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T08:27:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:30 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:27:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:30 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:27:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:30 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:27:30.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:30 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:27:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:30 smithi154 ceph-mon[36471]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T08:27:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:30 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:27:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:30 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:27:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:30 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:27:30.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:30 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:27:31.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:30 smithi049 ceph-mon[31942]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T08:27:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:30 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-03-16T08:27:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:30 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi154 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-16T08:27:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:30 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi129 to 174.1M 2024-03-16T08:27:31.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:30 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi129 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:31.744 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001570693, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000301689, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 6, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 162, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3344, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 184208, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.014561670, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.009536970, 2024-03-16T08:27:31.746 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002484353, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 160, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.014016247, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000087601 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 184208 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.747 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 219, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 209, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1436816, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 128544, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.045142786, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.013922082, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.037262887, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.004278542, 2024-03-16T08:27:31.748 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 208, 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.010429362, 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000050141 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:31.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1436816, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 128544 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:31.750 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:31.751 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:31.752 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:31.753 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148977, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-16T08:27:31.754 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:31.755 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 221, 2024-03-16T08:27:31.756 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1405959, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 221, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 221, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1405959, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:31.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:31.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:31.758 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-16T08:27:32.437 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-16T08:27:32.437 DEBUG:tasks.check_counter:Getting stats from g 2024-03-16T08:27:32.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-16T08:27:32.697 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:27:32.819 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:32 smithi049 ceph-mon[31942]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:32.819 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:32 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:32.819 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:32 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:32.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:32 smithi129 ceph-mon[36561]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:32.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:32 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:32.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:32 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:32.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:32 smithi154 ceph-mon[36471]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:32.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:32 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:32.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:32 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:33.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:27:33.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:33 smithi049 ceph-mon[31942]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:33.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:27:33.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:33 smithi129 ceph-mon[36561]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:27:33.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:33 smithi154 ceph-mon[36471]: from='mgr.15432 ' entity='mgr.x' 2024-03-16T08:27:34.377 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:34.377 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:34.378 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:34.378 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:34.378 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:34.378 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:34.378 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001370668, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000255796, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:34.379 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:34.380 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:34.381 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 6, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 159, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3344, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 183680, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:34.382 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.013915722, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.009170735, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002390971, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 157, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.017555135, 2024-03-16T08:27:34.383 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000111816 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.384 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 183680 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 215, 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 205, 2024-03-16T08:27:34.385 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1435728, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 125984, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.044873905, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.013976436, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.036013438, 2024-03-16T08:27:34.386 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.004231601, 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 204, 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.010297364, 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000050477 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.387 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1435728, 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 125984 2024-03-16T08:27:34.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-16T08:27:34.389 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:34.390 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:34.391 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:34.392 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:34.393 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:34.393 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:34.393 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:34.393 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:34.393 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:34.394 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148994, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 31, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:34.395 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:34.396 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-16T08:27:34.397 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 217, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1405439, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 217, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 217, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1405439, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:34.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:34.399 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:34.399 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:34.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:34 smithi049 ceph-mon[31942]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:34.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:34 smithi049 ceph-mon[31942]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:27:34.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:34 smithi049 ceph-mon[31942]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:27:34.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:34 smithi129 ceph-mon[36561]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:34.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:34 smithi129 ceph-mon[36561]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:27:34.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:34 smithi129 ceph-mon[36561]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:27:34.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:34 smithi154 ceph-mon[36471]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:34.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:34 smithi154 ceph-mon[36471]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-16T08:27:34.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:34 smithi154 ceph-mon[36471]: Unable to set osd_memory_target on smithi049 to 184761651: error parsing value: Value '184761651' is below minimum 939524096 2024-03-16T08:27:35.074 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-16T08:27:35.074 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-16T08:27:35.074 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-16T08:27:35.074 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-16T08:27:35.074 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-16T08:27:35.075 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-16T08:27:35.075 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-16T08:27:35.075 DEBUG:tasks.check_counter:Getting stats from j 2024-03-16T08:27:35.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-16T08:27:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T08:27:36.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:36 smithi049 ceph-mon[31942]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:36.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:36 smithi129 ceph-mon[36561]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:36.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:36 smithi154 ceph-mon[36471]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:37.039 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-16T08:27:37.039 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:37.039 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:37.039 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001383402, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000380129, 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:37.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.042 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 161, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 183936, 2024-03-16T08:27:37.043 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.014482964, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.010282257, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001726717, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:37.044 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 158, 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.009146543, 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000057889 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:37.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 183936 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 225, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 213, 2024-03-16T08:27:37.046 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1440384, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 130272, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.045652030, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.014226171, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.034942500, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.004326009, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 213, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.011732750, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000055083 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:37.047 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1440384, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 130272 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:37.048 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:37.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-16T08:27:37.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 149007, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:37.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-16T08:27:37.052 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 223, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1406219, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 223, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 223, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1406219, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-16T08:27:37.053 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-16T08:27:37.656 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-16T08:27:37.657 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-16T08:27:37.657 DEBUG:tasks.check_counter:Getting stats from b 2024-03-16T08:27:37.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-16T08:27:37.918 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:27:38.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:38 smithi129 ceph-mon[36561]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:38.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:38 smithi154 ceph-mon[36471]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:39.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:38 smithi049 ceph-mon[31942]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1510, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1353, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 507697, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 3352811, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 175, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-03-16T08:27:39.740 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.150141613, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.042580904, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.067986485, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.026509988, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1351, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.126312871, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000093495 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1343, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000201612, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000150 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.741 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 42224, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 976 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1490, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 949, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1912422, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 714646, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 12, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 6, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.123453669, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.030734960, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.068497052, 2024-03-16T08:27:39.742 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.036513758, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 948, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.242588177, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000255894 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 368, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000084841, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000230 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1542992, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 503984 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.743 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1720, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1038, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 765931, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 762768, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 33, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 6, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.132745298, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.032523213, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.052739077, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.043811143, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1036, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.156326703, 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000150894 2024-03-16T08:27:39.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2047, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000293543, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000143 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 350336, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 220224 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.745 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 728, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.214935459, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000295241 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 77, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.002999914, 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000038959 2024-03-16T08:27:39.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 160, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 160, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 160, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.144325296, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000902033 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 53, 2024-03-16T08:27:39.747 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 39, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 8, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 31, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 27, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 3, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 3, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 2, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 219, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 208, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-03-16T08:27:39.748 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 246, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 4, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 234, 2024-03-16T08:27:39.749 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 6, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 246, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 154, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 21, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 133, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 92, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 16, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 34, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:39.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 105, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 83, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 3, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 25 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 6, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 0, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 16, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 2, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 6, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 4, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 21, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-03-16T08:27:39.752 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 248, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 157, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 157, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 91, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 28, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 12, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 11, 2024-03-16T08:27:39.753 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 11, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 1, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 1, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 1, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4976498, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 5241175, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 0, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 211, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.360587664, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001708946 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 0 2024-03-16T08:27:39.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 29, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 54, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 25, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 20, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 20, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 0, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 39, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 43, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 4, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 3, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 22, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 19, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 54916, 2024-03-16T08:27:39.755 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 231804 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 212, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 160, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 32, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 8, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.006683983, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000835497 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-03-16T08:27:39.756 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 98, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.015258809, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000155702 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000574827, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000287413 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-03-16T08:27:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000259055, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000259055 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:39.759 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 10, 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001360819, 2024-03-16T08:27:39.760 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000136081 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 8, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.014430311, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001803788 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.761 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 10, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.040021189, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.004002118 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000149645, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000149645 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.762 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.004117063, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002058531 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.002021054, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001010527 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.015350257, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.007675128 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.015397782, 2024-03-16T08:27:39.764 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.003849445 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000803577, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000803577 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.765 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.766 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 7, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.018390566, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002627223 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.009506359, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002376589 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.767 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 2, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 5, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 3, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 2, 2024-03-16T08:27:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 27, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 13, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 608, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:39.770 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:39.771 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 346458, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 141, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 616, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 7, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 166, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:39.772 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 134701, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 1242, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 1253, 2024-03-16T08:27:39.773 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 1140782, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 32, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 1253, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1253, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 17.810617907, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.014214379 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1253, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 1407 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.774 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 1253, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 290, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 963, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 33, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 28, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 23, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 213, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 56, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 579, 2024-03-16T08:27:39.775 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 267, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 56, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-03-16T08:27:39.776 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-03-16T08:27:39.777 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 97, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 12, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 12, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 12, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 141, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 0, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 11 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 1, 2024-03-16T08:27:39.778 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 64, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 3, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 67, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 4, 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:39.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 4716, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 2708930, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 4716, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 4716, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2708930, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.780 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 1253, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 1141218, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 1253, 2024-03-16T08:27:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1141218, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 1253, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 1253, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 1253, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1253, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:39.783 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 4, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 436, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 4, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 436, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:39.784 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x5576822a4500": { 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 248, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1046871, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 248, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 211, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1046871, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:39.785 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:39.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:39.786 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=6 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-16T08:27:40.542 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-16T08:27:40.543 DEBUG:tasks.check_counter:Getting stats from e 2024-03-16T08:27:40.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-16T08:27:40.808 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:27:40.843 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:40 smithi129 ceph-mon[36561]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:40.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:40 smithi154 ceph-mon[36471]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:41.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:40 smithi049 ceph-mon[31942]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 239, 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 228, 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1462594, 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 132916, 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:42.552 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.034605357, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.008255426, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.032369859, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003633417, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 227, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.007350760, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000032382 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.553 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1461760, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 132592 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 632, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 791, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 155322, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 645241, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 7, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 5, 2024-03-16T08:27:42.554 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.048326151, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.015956383, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.010839464, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.016691825, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 788, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.027414374, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000034789 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.555 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 444896 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1518, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1518, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 457534, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 481958, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.095413288, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.032080867, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.024267302, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.031314040, 2024-03-16T08:27:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1517, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.033991172, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000022406 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-03-16T08:27:42.557 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3634, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.073196294, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000020142 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.558 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 0, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 0, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-03-16T08:27:42.559 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 1, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 0, 2024-03-16T08:27:42.560 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:42.561 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-03-16T08:27:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-03-16T08:27:42.563 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:42.564 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 7, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 0, 2024-03-16T08:27:42.565 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4196347, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 4196206, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.566 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 7 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 3, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 3, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 1, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 1, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 0, 2024-03-16T08:27:42.567 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 47636, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 215420 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.570 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.571 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-03-16T08:27:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.574 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-03-16T08:27:42.576 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-03-16T08:27:42.577 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.578 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-03-16T08:27:42.579 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 1, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 3, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 2, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 3, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 5137, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:42.580 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:42.581 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:42.582 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 189433, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 52, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 166, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:42.583 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 8464, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 13, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 2145, 2024-03-16T08:27:42.584 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 21, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 3, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 2145, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2145, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 1.281973197, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000597656 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2145, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 2146 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 2145, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 2145, 2024-03-16T08:27:42.585 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 1382, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 761, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-03-16T08:27:42.586 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 1, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-03-16T08:27:42.587 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 97, 2024-03-16T08:27:42.588 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 11, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 5, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 5, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 2, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 0 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-03-16T08:27:42.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:42.590 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 2385, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1845585, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 2385, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2385, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1845585, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 2145, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 2048, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2145, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2048, 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:42.592 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 2145, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 2145, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2145, 2024-03-16T08:27:42.593 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2145, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:42.594 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x5610cc3826e0": { 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:42.595 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:42.597 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-03-16T08:27:42.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:42 smithi129 ceph-mon[36561]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:42.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:42 smithi154 ceph-mon[36471]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:43.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:42 smithi049 ceph-mon[31942]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-16T08:27:43.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T08:27:43.228 DEBUG:tasks.check_counter:Getting stats from h 2024-03-16T08:27:43.228 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-16T08:27:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:27:44.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:44 smithi129 ceph-mon[36561]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:44.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:44 smithi154 ceph-mon[36471]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:45.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:44 smithi049 ceph-mon[31942]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 541, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 738, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 155421, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 549337, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 9, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.053088913, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.024572559, 2024-03-16T08:27:45.191 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.023034307, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003051225, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 740, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.038449817, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000051959 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1091, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000171548, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000157 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 315104 2024-03-16T08:27:45.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 83, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 143, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 19025, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 597359, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 49, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 6, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.018024769, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.003187818, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.013188563, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000256272, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 141, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.014884617, 2024-03-16T08:27:45.193 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000105564 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 118, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000014663, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000124 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 9744, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 145120 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 330, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 321, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1492439, 2024-03-16T08:27:45.194 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 251055, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 15, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 6, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.040477406, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009763087, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.039325747, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003655464, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 316, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.014783230, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000046782 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 93, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000014420, 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000155 2024-03-16T08:27:45.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1471184, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 136272 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 51, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001472326, 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000028869 2024-03-16T08:27:45.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000007565, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000003782 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 10, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 6, 2024-03-16T08:27:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 6, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001534813, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000255802 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 4, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 11, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 16, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-03-16T08:27:45.198 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 16, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 13, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 5, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 1, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 1, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 3, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 21, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 13, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 1, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 4, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-03-16T08:27:45.199 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 4, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 234, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 6, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:45.200 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.201 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 9, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 6, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 8, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-03-16T08:27:45.202 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-03-16T08:27:45.203 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 7, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 7, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 2, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4196347, 2024-03-16T08:27:45.204 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 0, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 6, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.012394116, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002065686 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 0 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 20, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 22, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 2, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 19, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 21, 2024-03-16T08:27:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 2, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 16, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 21, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 5, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 1, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 3, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 2, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 52764, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 223612 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 11, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 10, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 16, 2024-03-16T08:27:45.206 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000858732, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000214683 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.207 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000676081, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000338040 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.210 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.211 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.212 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.213 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.214 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.216 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.217 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 1, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 3, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 2, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 1, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 4, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 1, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 580, 2024-03-16T08:27:45.218 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:45.219 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:45.220 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 256535, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 99, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 176, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 2, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 166, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 83453, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 336, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-03-16T08:27:45.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 45, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 12580, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 29, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 45, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 45, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.240994852, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.005355441 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 45, 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 69 2024-03-16T08:27:45.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 45, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 45, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 10, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 6, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 8, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 6, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-03-16T08:27:45.224 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 20, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-03-16T08:27:45.225 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 97, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 11, 2024-03-16T08:27:45.226 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 9, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 9, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 17, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 0, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 2 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 1, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.227 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 950, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1563745, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:45.228 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 950, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 950, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1563745, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:45.229 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 45, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 12580, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 45, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 12580, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:45.230 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 45, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 45, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 45, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 45, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.231 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x5639a6f4e500": { 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 7, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 2043, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 7, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 6, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2043, 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:45.233 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:45.234 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:45.234 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:45.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:45.234 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=9 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-16T08:27:45.982 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T08:27:45.983 DEBUG:tasks.check_counter:Getting stats from k 2024-03-16T08:27:45.983 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-16T08:27:46.232 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T08:27:46.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:46 smithi129 ceph-mon[36561]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:46.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:46 smithi154 ceph-mon[36471]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:47.038 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:46 smithi049 ceph-mon[31942]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:47.981 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 225, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 215, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1438512, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 131904, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.036676451, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.008190618, 2024-03-16T08:27:47.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.034898601, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003793528, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 214, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.006905381, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000032268 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1438512, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 131904 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.983 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 6, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 166, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 3344, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 184912, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.008275149, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.004489425, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.002380498, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 164, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.014224917, 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000086737 2024-03-16T08:27:47.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 184912 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:47.985 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.000398826, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000322429, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:47.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:47.987 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:47.988 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:47.989 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 149057, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 32, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:47.990 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 227, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1406803, 2024-03-16T08:27:47.991 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 227, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 227, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1406803, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-03-16T08:27:47.992 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-03-16T08:27:48.648 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:48 smithi129 ceph-mon[36561]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:48.681 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-16T08:27:48.681 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-16T08:27:48.681 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-16T08:27:48.682 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-16T08:27:48.682 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-16T08:27:48.682 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-16T08:27:48.682 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-16T08:27:48.682 DEBUG:tasks.check_counter:Getting stats from c 2024-03-16T08:27:48.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-16T08:27:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:27:48.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:48 smithi154 ceph-mon[36471]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:49.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:48 smithi049 ceph-mon[31942]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:50.619 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:50 smithi154 ceph-mon[36471]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:50.746 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-16T08:27:50.746 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:50.746 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1648, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1648, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1535818, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 523250, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 2, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.095735511, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.027746695, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.026997223, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.032136591, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1647, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.038941375, 2024-03-16T08:27:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000023643 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 244, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 233, 2024-03-16T08:27:50.748 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1463912, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 134250, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.032832176, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.008757551, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.029952745, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003564624, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 232, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.010326178, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000044509 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:50.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1462256, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 132912 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 746, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 906, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 183415, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 684897, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-03-16T08:27:50.750 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.057394859, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.020665709, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.012565808, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.018598829, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 903, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.025049919, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000027740 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 448304 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 3981, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.075912186, 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000019068 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-03-16T08:27:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 31, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 7, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 24, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 22, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 2, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-03-16T08:27:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T08:27:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-03-16T08:27:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-03-16T08:27:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 248, 2024-03-16T08:27:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 12, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4976498, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 5241175, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 5241063, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 248 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 20, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 24, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 4, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 11, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 11, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 0, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 31, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 31, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 0, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 50496, 2024-03-16T08:27:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215420 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-03-16T08:27:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-03-16T08:27:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.765 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-03-16T08:27:50.766 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.767 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.768 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-03-16T08:27:50.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.772 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 2, 2024-03-16T08:27:50.773 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 5, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 3, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 4, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 3644, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:50.774 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:50.775 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-03-16T08:27:50.776 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 189672, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 59, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 166, 2024-03-16T08:27:50.777 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 91909, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 752, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 2392, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 26, 2024-03-16T08:27:50.778 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 4, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 2392, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2392, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.979974418, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000409688 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2392, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 2393 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 2392, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 2392, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-03-16T08:27:50.779 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 1506, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 883, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 2, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:50.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-03-16T08:27:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 97, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 11, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 5, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 5, 2024-03-16T08:27:50.782 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-03-16T08:27:50.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:50.784 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 2634, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 2930772, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 2634, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2634, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 2930772, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.785 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 2392, 2024-03-16T08:27:50.786 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 1046881, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2392, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1046881, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:50.787 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 2392, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 2392, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2392, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 2392, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:50.789 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55dccece86e0": { 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:50.790 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:50.791 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-16T08:27:50.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:50 smithi129 ceph-mon[36561]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:51.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:50 smithi049 ceph-mon[31942]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-16T08:27:51.440 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T08:27:51.440 DEBUG:tasks.check_counter:Getting stats from f 2024-03-16T08:27:51.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-16T08:27:51.687 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:27:52.832 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:52 smithi154 ceph-mon[36471]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:52.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:52 smithi129 ceph-mon[36561]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:53.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:52 smithi049 ceph-mon[31942]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:53.443 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-16T08:27:53.443 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:53.443 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1006, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 995, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1719680, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 377302, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.085270251, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.023561845, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.044754834, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.021155817, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 994, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.029493850, 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000029671 2024-03-16T08:27:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1462640, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 134032 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 4, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 165, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 876, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 449178, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-03-16T08:27:53.445 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.010074869, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004422507, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001723481, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000078878, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 162, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.014502540, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000089521 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 448480 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1458, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1458, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 375944, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 462827, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.073924915, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.021889436, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.019263363, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.028402413, 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:53.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1457, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.031484432, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000021609 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-03-16T08:27:53.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 3756, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.070390291, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000018740 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-03-16T08:27:53.450 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 1, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:53.451 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:53.452 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-03-16T08:27:53.453 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-03-16T08:27:53.454 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-03-16T08:27:53.455 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 8, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-03-16T08:27:53.456 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 4196655, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 4196514, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 8 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 3, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 3, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 0, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 1, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 1, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 0, 2024-03-16T08:27:53.457 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 49676, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215420 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-03-16T08:27:53.458 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.459 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.461 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-03-16T08:27:53.462 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.463 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-03-16T08:27:53.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.465 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.467 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.468 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 2, 2024-03-16T08:27:53.469 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 4, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 3, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 2893, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:53.470 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:53.471 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:53.472 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 181343, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 47, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 166, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 8464, 2024-03-16T08:27:53.473 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 13, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 2221, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 3, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 2221, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2221, 2024-03-16T08:27:53.474 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.044972764, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000470496 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2221, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 2222 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 2221, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 2221, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 1444, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-03-16T08:27:53.475 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 775, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:53.476 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-03-16T08:27:53.477 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 97, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 11, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 4, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 4, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-03-16T08:27:53.478 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-03-16T08:27:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 2464, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1859369, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:53.480 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 2464, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2464, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1859369, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:53.481 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 2221, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 2356, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2221, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 2356, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 2221, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 2221, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 2221, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 2221, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-03-16T08:27:53.484 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x5606c194a6e0": { 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:53.485 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:53.486 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-16T08:27:54.285 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T08:27:54.285 DEBUG:tasks.check_counter:Getting stats from i 2024-03-16T08:27:54.286 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-16T08:27:54.534 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:27:54.839 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:54 smithi154 ceph-mon[36471]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:54.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:54 smithi129 ceph-mon[36561]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:55.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:54 smithi049 ceph-mon[31942]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T08:27:56.283 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 84, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 122, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 16369, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 440550, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 49, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 9, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.020380419, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.003037606, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.012734558, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000539018, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 113, 2024-03-16T08:27:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.018897397, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000167233 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 89, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000016954, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000190 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 2864, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 2560 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 551, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 795, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 158186, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 839084, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 13, 2024-03-16T08:27:56.285 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.046824356, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.019274655, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.024186078, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003167410, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 797, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.036953296, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000046365 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1148, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000183779, 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000160 2024-03-16T08:27:56.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 455072 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 336, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 317, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1499774, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 234841, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.039332068, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.009808137, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.036744901, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003894768, 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:56.287 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 319, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.018133128, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000056843 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 100, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000014318, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000143 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1478560, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 137840 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.288 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 48, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.001684624, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000035096 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000006391, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000003195 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-03-16T08:27:56.289 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 4, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 2, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000365481, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000182740 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 2, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 11, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 16, 2024-03-16T08:27:56.290 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 16, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 13, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 4, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 1, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 1, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 3, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 11, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 5, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 4, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-03-16T08:27:56.291 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T08:27:56.292 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 2 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 9, 2024-03-16T08:27:56.293 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 6, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 8, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-03-16T08:27:56.294 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 8, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 8, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-03-16T08:27:56.295 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 2, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 4196655, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.010372132, 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001481733 2024-03-16T08:27:56.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 20, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 22, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 2, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 19, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 22, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 3, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 16, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 20, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 4, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 1, 2024-03-16T08:27:56.297 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 2, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 1, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 50552, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 223612 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 5, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 4, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 25, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000214953, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000214953 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.299 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000150528, 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000150528 2024-03-16T08:27:56.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-03-16T08:27:56.301 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-03-16T08:27:56.304 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.306 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 2, 2024-03-16T08:27:56.308 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 4, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 2, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 1, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 615, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:56.309 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:56.310 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 247829, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 92, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:56.311 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7944, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 166, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 83405, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 330, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-03-16T08:27:56.312 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 45, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 12833, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 28, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 45, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 45, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.221994028, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004933200 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-03-16T08:27:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 45, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 67 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 45, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 45, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 7, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 9, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-03-16T08:27:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 6, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-03-16T08:27:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 97, 2024-03-16T08:27:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 11, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 9, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 9, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 14, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 1 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-03-16T08:27:56.317 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:56.318 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 967, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1569988, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 967, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 967, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1569988, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:56.319 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 45, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 12833, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 45, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 12833, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.320 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 45, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 45, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 45, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 45, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.321 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:56.322 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55f7cce80500": { 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 8, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 2351, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 8, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 7, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 2351, 2024-03-16T08:27:56.323 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:56.324 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-16T08:27:56.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:56 smithi129 ceph-mon[36561]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:56.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:56 smithi154 ceph-mon[36471]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:57.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:56 smithi049 ceph-mon[31942]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=9 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-16T08:27:57.065 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T08:27:57.065 DEBUG:tasks.check_counter:Getting stats from l 2024-03-16T08:27:57.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 053f2308-e36c-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-16T08:27:57.308 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T08:27:58.811 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:27:58 smithi154 ceph-mon[36471]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:27:58.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:27:58 smithi129 ceph-mon[36561]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:27:59.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:27:58 smithi049 ceph-mon[31942]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:27:59.087 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-16T08:27:59.087 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.001063401, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000265611, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:59.088 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.089 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 229, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 219, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1439600, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 134464, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.033656861, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.008263003, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.031931538, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003606272, 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:59.090 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 218, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.007995145, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000036674 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1439600, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 134464 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.091 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 6, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 169, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 3344, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 185440, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.008066810, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004641894, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.002049503, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T08:27:59.092 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 167, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.010442131, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000062527 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 185440 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-03-16T08:27:59.093 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T08:27:59.094 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T08:27:59.095 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T08:27:59.096 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 149057, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 32, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-03-16T08:27:59.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-03-16T08:27:59.098 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 231, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1407323, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 231, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 231, 2024-03-16T08:27:59.099 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1407323, 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-16T08:27:59.100 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-16T08:27:59.740 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-16T08:27:59.741 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-16T08:27:59.741 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-16T08:27:59.751 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T08:27:59.751 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:27:59.751 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T08:27:59.773 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-16T08:27:59.773 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T08:27:59.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:27:59.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- 172.21.15.49:0/1135726110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 msgr2=0x7f8fd0106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:59.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/1135726110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd0106a70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009930 tx=0x7f8fc402bc30 comp rx=0 tx=0).stop 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- 172.21.15.49:0/1135726110 shutdown_connections 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/1135726110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 0x7f8fd010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/1135726110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 0x7f8fd0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/1135726110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd0106a70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- 172.21.15.49:0/1135726110 >> 172.21.15.49:0/1135726110 conn(0x7f8fd0076a60 msgr2=0x7f8fd0076e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- 172.21.15.49:0/1135726110 shutdown_connections 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- 172.21.15.49:0/1135726110 wait complete. 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 Processor -- start 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.992+0000 7f8fd6e61640 1 -- start start 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 0x7f8fd01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 0x7f8fd01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110aa0 con 0x7f8fd0106690 2024-03-16T08:27:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110920 con 0x7f8fd0101040 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd6e61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110c20 con 0x7f8fd0100680 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd4bfd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 0x7f8fd01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33612/0 (socket says 172.21.15.49:33612) 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 -- 172.21.15.49:0/2340887869 learned_addr learned my addr 172.21.15.49:0/2340887869 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fcffff640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 0x7f8fd01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 -- 172.21.15.49:0/2340887869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 msgr2=0x7f8fd01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 0x7f8fd01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 -- 172.21.15.49:0/2340887869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 msgr2=0x7f8fd01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 0x7f8fd01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 -- 172.21.15.49:0/2340887869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019d0a0 con 0x7f8fd0106690 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.993+0000 7f8fd53fe640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800eb10 tx=0x7f8fc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.994+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004b20 con 0x7f8fd0106690 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.994+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019d390 con 0x7f8fd0106690 2024-03-16T08:27:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.994+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004cc0 con 0x7f8fd0106690 2024-03-16T08:27:59.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.994+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8010760 con 0x7f8fd0106690 2024-03-16T08:27:59.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.994+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a4c20 con 0x7f8fd0106690 2024-03-16T08:27:59.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.995+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd0101dc0 con 0x7f8fd0106690 2024-03-16T08:27:59.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.996+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8fc8005020 con 0x7f8fd0106690 2024-03-16T08:27:59.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.996+0000 7f8fcdffb640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 0x7f8fc00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:27:59.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.996+0000 7f8fd4bfd640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 0x7f8fc00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:27:59.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.996+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f8fc80cf350 con 0x7f8fd0106690 2024-03-16T08:27:59.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:27:59.997+0000 7f8fd4bfd640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 0x7f8fc00a4e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005ab0 tx=0x7f8fc4005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.001+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8fc809c0d0 con 0x7f8fd0106690 2024-03-16T08:28:00.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.184+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fd0199400 con 0x7f8fd0106690 2024-03-16T08:28:00.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.186+0000 7f8fcdffb640 1 -- 172.21.15.49:0/2340887869 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23055 (secure 0 0 0) 0x7f8fc809b9d0 con 0x7f8fd0106690 2024-03-16T08:28:00.187 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:28:00.188 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":97,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:27:14.055824+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/1561370229":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3786569188":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3762364594":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/236450725":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 msgr2=0x7f8fc00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 0x7f8fc00a4e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005ab0 tx=0x7f8fc4005fd0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 msgr2=0x7f8fd019c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800eb10 tx=0x7f8fc800c2d0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 shutdown_connections 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8fc00a2970 0x7f8fc00a4e30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fd0106690 0x7f8fd019c970 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0101040 0x7f8fd01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 --2- 172.21.15.49:0/2340887869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0100680 0x7f8fd01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 >> 172.21.15.49:0/2340887869 conn(0x7f8fd0076a60 msgr2=0x7f8fd006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 shutdown_connections 2024-03-16T08:28:00.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.188+0000 7f8fd6e61640 1 -- 172.21.15.49:0/2340887869 wait complete. 2024-03-16T08:28:00.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:10:28.036537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T08:14:59.603963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T08:15:00.940257+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T08:28:00.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 -- 172.21.15.49:0/1538787312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0102140 msgr2=0x7f4fc010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 --2- 172.21.15.49:0/1538787312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0102140 0x7f4fc010e650 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009cd0 tx=0x7f4fb802c0f0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 -- 172.21.15.49:0/1538787312 shutdown_connections 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 --2- 172.21.15.49:0/1538787312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0102140 0x7f4fc010e650 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 --2- 172.21.15.49:0/1538787312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0101780 0x7f4fc0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 --2- 172.21.15.49:0/1538787312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0107790 0x7f4fc0107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 -- 172.21.15.49:0/1538787312 >> 172.21.15.49:0/1538787312 conn(0x7f4fc00fd210 msgr2=0x7f4fc00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 -- 172.21.15.49:0/1538787312 shutdown_connections 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.310+0000 7f4fc6110640 1 -- 172.21.15.49:0/1538787312 wait complete. 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 Processor -- start 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 -- start start 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0101780 0x7f4fc0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 0x7f4fc0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0111ba0 con 0x7f4fc0107790 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0111a20 con 0x7f4fc0102140 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fc6110640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0111d20 con 0x7f4fc0101780 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.311+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fb6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 0x7f4fc0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33624/0 (socket says 172.21.15.49:33624) 2024-03-16T08:28:00.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0101780 0x7f4fc0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 -- 172.21.15.49:0/4070260007 learned_addr learned my addr 172.21.15.49:0/4070260007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 -- 172.21.15.49:0/4070260007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0101780 msgr2=0x7f4fc0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0101780 0x7f4fc0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 -- 172.21.15.49:0/4070260007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 msgr2=0x7f4fc0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 0x7f4fc0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 -- 172.21.15.49:0/4070260007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc019d0a0 con 0x7f4fc0107790 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fb6ffd640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 0x7f4fc0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbffff640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80348f0 tx=0x7f4fb8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8041020 con 0x7f4fc0107790 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8032e80 con 0x7f4fc0107790 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc019d330 con 0x7f4fc0107790 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8039040 con 0x7f4fc0107790 2024-03-16T08:28:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.312+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc01a4c20 con 0x7f4fc0107790 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.313+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc0068a20 con 0x7f4fc0107790 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.314+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4fb8044020 con 0x7f4fc0107790 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.315+0000 7f4fbd7fa640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 0x7f4fac0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.315+0000 7f4fbf7fe640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 0x7f4fac0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.315+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f4fb80f08e0 con 0x7f4fc0107790 2024-03-16T08:28:00.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.316+0000 7f4fbf7fe640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 0x7f4fac0a4e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009b50 tx=0x7f4fb0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.320+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4fb804b020 con 0x7f4fc0107790 2024-03-16T08:28:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.502+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fc0109cf0 con 0x7f4fc0107790 2024-03-16T08:28:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.503+0000 7f4fbd7fa640 1 -- 172.21.15.49:0/4070260007 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f4fb80bd770 con 0x7f4fc0107790 2024-03-16T08:28:00.504 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-16T08:28:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 msgr2=0x7f4fac0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 0x7f4fac0a4e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009b50 tx=0x7f4fb0005f90 comp rx=0 tx=0).stop 2024-03-16T08:28:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 msgr2=0x7f4fc019c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80348f0 tx=0x7f4fb8032c90 comp rx=0 tx=0).stop 2024-03-16T08:28:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 shutdown_connections 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f4fac0a2970 0x7f4fac0a4e30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fc0107790 0x7f4fc019c970 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fc0102140 0x7f4fc0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 --2- 172.21.15.49:0/4070260007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc0101780 0x7f4fc0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 >> 172.21.15.49:0/4070260007 conn(0x7f4fc00fd210 msgr2=0x7f4fc00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 shutdown_connections 2024-03-16T08:28:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.505+0000 7f4fc6110640 1 -- 172.21.15.49:0/4070260007 wait complete. 2024-03-16T08:28:00.517 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T08:28:00.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 -- 172.21.15.49:0/1761145878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c106710 msgr2=0x7fd61c106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 --2- 172.21.15.49:0/1761145878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c106710 0x7fd61c106af0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009930 tx=0x7fd60c02bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:00.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 -- 172.21.15.49:0/1761145878 shutdown_connections 2024-03-16T08:28:00.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 --2- 172.21.15.49:0/1761145878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c1010c0 0x7fd61c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 --2- 172.21.15.49:0/1761145878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c100700 0x7fd61c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 --2- 172.21.15.49:0/1761145878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c106710 0x7fd61c106af0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 -- 172.21.15.49:0/1761145878 >> 172.21.15.49:0/1761145878 conn(0x7fd61c077840 msgr2=0x7fd61c077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 -- 172.21.15.49:0/1761145878 shutdown_connections 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 -- 172.21.15.49:0/1761145878 wait complete. 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.627+0000 7fd6235b3640 1 Processor -- start 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 -- start start 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c100700 0x7fd61c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c106710 0x7fd61c1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c110b20 con 0x7fd61c1010c0 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1109a0 con 0x7fd61c106710 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6235b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c110ca0 con 0x7fd61c100700 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33630/0 (socket says 172.21.15.49:33630) 2024-03-16T08:28:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 -- 172.21.15.49:0/2147849093 learned_addr learned my addr 172.21.15.49:0/2147849093 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd621b50640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c106710 0x7fd61c1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd62134f640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c100700 0x7fd61c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 -- 172.21.15.49:0/2147849093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c100700 msgr2=0x7fd61c198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c100700 0x7fd61c198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 -- 172.21.15.49:0/2147849093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c106710 msgr2=0x7fd61c1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c106710 0x7fd61c1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 -- 172.21.15.49:0/2147849093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1a4540 con 0x7fd61c1010c0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd620b4e640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.628+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cd80 con 0x7fd61c1010c0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.629+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a4710 con 0x7fd61c1010c0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.629+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd618010910 con 0x7fd61c1010c0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.629+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a4c20 con 0x7fd61c1010c0 2024-03-16T08:28:00.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.629+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618018910 con 0x7fd61c1010c0 2024-03-16T08:28:00.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.630+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c068a20 con 0x7fd61c1010c0 2024-03-16T08:28:00.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.631+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd618010ac0 con 0x7fd61c1010c0 2024-03-16T08:28:00.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.631+0000 7fd6127fc640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 0x7fd6080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.631+0000 7fd62134f640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 0x7fd6080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.632+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7fd6180cf1b0 con 0x7fd61c1010c0 2024-03-16T08:28:00.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.632+0000 7fd62134f640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 0x7fd6080a4e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005ab0 tx=0x7fd60c005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.636+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd61809b8e0 con 0x7fd61c1010c0 2024-03-16T08:28:00.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:00 smithi049 ceph-mon[31942]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:28:00.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2340887869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:00.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:00 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/4070260007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:28:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.819+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd61c02d1b0 con 0x7fd61c1010c0 2024-03-16T08:28:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.819+0000 7fd6127fc640 1 -- 172.21.15.49:0/2147849093 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fd618084e20 con 0x7fd61c1010c0 2024-03-16T08:28:00.821 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 msgr2=0x7fd6080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 0x7fd6080a4e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005ab0 tx=0x7fd60c005fd0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 msgr2=0x7fd61c1994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 shutdown_connections 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fd6080a2970 0x7fd6080a4e30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd61c106710 0x7fd61c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c1010c0 0x7fd61c1994c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 --2- 172.21.15.49:0/2147849093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c100700 0x7fd61c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.821+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 >> 172.21.15.49:0/2147849093 conn(0x7fd61c077840 msgr2=0x7fd61c103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.822+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 shutdown_connections 2024-03-16T08:28:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.822+0000 7fd6235b3640 1 -- 172.21.15.49:0/2147849093 wait complete. 2024-03-16T08:28:00.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T08:28:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:00 smithi129 ceph-mon[36561]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:28:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2340887869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:00.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:00 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/4070260007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:28:00.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 -- 172.21.15.49:0/3364910211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 msgr2=0x7f99f8101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 --2- 172.21.15.49:0/3364910211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f8101be0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009930 tx=0x7f99e802bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:00.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 -- 172.21.15.49:0/3364910211 shutdown_connections 2024-03-16T08:28:00.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 --2- 172.21.15.49:0/3364910211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8102120 0x7f99f810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 --2- 172.21.15.49:0/3364910211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f8101be0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 --2- 172.21.15.49:0/3364910211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8107770 0x7f99f8107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 -- 172.21.15.49:0/3364910211 >> 172.21.15.49:0/3364910211 conn(0x7f99f80fd270 msgr2=0x7f99f80ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 -- 172.21.15.49:0/3364910211 shutdown_connections 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 -- 172.21.15.49:0/3364910211 wait complete. 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.949+0000 7f9a007df640 1 Processor -- start 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 -- start start 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8102120 0x7f99f8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8107770 0x7f99f819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8111b80 con 0x7f99f8101760 2024-03-16T08:28:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8111a00 con 0x7f99f8107770 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f9a007df640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8111d00 con 0x7f99f8102120 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33642/0 (socket says 172.21.15.49:33642) 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 -- 172.21.15.49:0/846172590 learned_addr learned my addr 172.21.15.49:0/846172590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fed7c640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8107770 0x7f99f819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fdd7a640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8102120 0x7f99f8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 -- 172.21.15.49:0/846172590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8102120 msgr2=0x7f99f8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8102120 0x7f99f8198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 -- 172.21.15.49:0/846172590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8107770 msgr2=0x7f99f819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8107770 0x7f99f819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 -- 172.21.15.49:0/846172590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f819d0b0 con 0x7f99f8101760 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.950+0000 7f99fe57b640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f99f400e9f0 tx=0x7f99f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.951+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400cd80 con 0x7f99f8101760 2024-03-16T08:28:00.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.951+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f4010910 con 0x7f99f8101760 2024-03-16T08:28:00.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.951+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f819d3a0 con 0x7f99f8101760 2024-03-16T08:28:00.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.951+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4018990 con 0x7f99f8101760 2024-03-16T08:28:00.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.951+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f81a4c30 con 0x7f99f8101760 2024-03-16T08:28:00.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.952+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f80685d0 con 0x7f99f8101760 2024-03-16T08:28:00.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.953+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f99f4010ac0 con 0x7f99f8101760 2024-03-16T08:28:00.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.954+0000 7f99ef7fe640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 0x7f99e40a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:00.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.954+0000 7f99fdd7a640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 0x7f99e40a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:00.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.954+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f99f40cefc0 con 0x7f99f8101760 2024-03-16T08:28:00.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.955+0000 7f99fdd7a640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 0x7f99e40a4f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:00.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:00.961+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f99f409b6f0 con 0x7f99f8101760 2024-03-16T08:28:00.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:00 smithi154 ceph-mon[36471]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-16T08:28:00.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2340887869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:00.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:00 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/4070260007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.151+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f99f802d1b0 con 0x7f99f8101760 2024-03-16T08:28:01.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.151+0000 7f99ef7fe640 1 -- 172.21.15.49:0/846172590 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f99f409f790 con 0x7f99f8101760 2024-03-16T08:28:01.152 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 msgr2=0x7f99e40a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 0x7f99e40a4f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8005fd0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 msgr2=0x7f99f81980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f99f400e9f0 tx=0x7f99f400eeb0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 shutdown_connections 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f99e40a2ac0 0x7f99e40a4f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99f8107770 0x7f99f819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99f8102120 0x7f99f8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 --2- 172.21.15.49:0/846172590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f8101760 0x7f99f81980c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 >> 172.21.15.49:0/846172590 conn(0x7f99f80fd270 msgr2=0x7f99f80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:01.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 shutdown_connections 2024-03-16T08:28:01.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.153+0000 7f9a007df640 1 -- 172.21.15.49:0/846172590 wait complete. 2024-03-16T08:28:01.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 -- 172.21.15.49:0/649997869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860107780 msgr2=0x7f5860107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 --2- 172.21.15.49:0/649997869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860107780 0x7f5860107b60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5850009cd0 tx=0x7f585002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 -- 172.21.15.49:0/649997869 shutdown_connections 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 --2- 172.21.15.49:0/649997869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860102130 0x7f586010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 --2- 172.21.15.49:0/649997869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 0x7f5860101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 --2- 172.21.15.49:0/649997869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860107780 0x7f5860107b60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 -- 172.21.15.49:0/649997869 >> 172.21.15.49:0/649997869 conn(0x7f5860078be0 msgr2=0x7f5860078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 -- 172.21.15.49:0/649997869 shutdown_connections 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 -- 172.21.15.49:0/649997869 wait complete. 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.274+0000 7f5865e59640 1 Processor -- start 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 -- start start 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 0x7f5860198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860107780 0x7f586019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111b90 con 0x7f5860102130 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111a10 con 0x7f5860107780 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111d10 con 0x7f5860101770 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33650/0 (socket says 172.21.15.49:33650) 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 -- 172.21.15.49:0/3646852589 learned_addr learned my addr 172.21.15.49:0/3646852589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:01.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585ffff640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860107780 0x7f586019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585f7fe640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 0x7f5860198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 -- 172.21.15.49:0/3646852589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 msgr2=0x7f5860198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 0x7f5860198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 -- 172.21.15.49:0/3646852589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860107780 msgr2=0x7f586019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860107780 0x7f586019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 -- 172.21.15.49:0/3646852589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586019d060 con 0x7f5860102130 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585effd640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9f0 tx=0x7f584800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584800ccc0 con 0x7f5860102130 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584800ce60 con 0x7f5860102130 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.275+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586019d350 con 0x7f5860102130 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.276+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848010690 con 0x7f5860102130 2024-03-16T08:28:01.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.276+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601a4be0 con 0x7f5860102130 2024-03-16T08:28:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.276+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5860068a20 con 0x7f5860102130 2024-03-16T08:28:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.278+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f58480046c0 con 0x7f5860102130 2024-03-16T08:28:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.278+0000 7f585cff9640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 0x7f584c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.278+0000 7f585f7fe640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 0x7f584c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.278+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f58480cf1a0 con 0x7f5860102130 2024-03-16T08:28:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.279+0000 7f585f7fe640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 0x7f584c0a4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5850004af0 tx=0x7f5850005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.282+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f584809bf20 con 0x7f5860102130 2024-03-16T08:28:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.480+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f586002d1b0 con 0x7f5860102130 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.480+0000 7f585cff9640 1 -- 172.21.15.49:0/3646852589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v97) ==== 81+0+944 (secure 0 0 0) 0x7f584809b820 con 0x7f5860102130 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2256169095 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2256169095 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1561370229 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3786569188 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3066712091 2024-03-17T08:08:03.498489+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/372564765 2024-03-17T08:08:03.498489+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1466047614 2024-03-17T08:07:36.305451+0000 2024-03-16T08:28:01.482 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1726594106 2024-03-17T08:07:36.305451+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3762364594 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2659026035 2024-03-17T08:08:03.498489+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1374126004 2024-03-17T08:07:36.305451+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1374126004 2024-03-17T08:07:36.305451+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/236450725 2024-03-17T08:17:12.732236+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3199936220 2024-03-17T08:07:36.305451+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2659026035 2024-03-17T08:08:03.498489+0000 2024-03-16T08:28:01.483 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3531600398 2024-03-17T08:08:03.498489+0000 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:listed 16 entries 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 msgr2=0x7f584c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 0x7f584c0a4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5850004af0 tx=0x7f5850005c50 comp rx=0 tx=0).stop 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 msgr2=0x7f58601985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9f0 tx=0x7f584800edf0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 shutdown_connections 2024-03-16T08:28:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f584c0a29f0 0x7f584c0a4eb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5860107780 0x7f586019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5860102130 0x7f58601985b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 --2- 172.21.15.49:0/3646852589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860101770 0x7f5860198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 >> 172.21.15.49:0/3646852589 conn(0x7f5860078be0 msgr2=0x7f5860104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 shutdown_connections 2024-03-16T08:28:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.482+0000 7f5865e59640 1 -- 172.21.15.49:0/3646852589 wait complete. 2024-03-16T08:28:01.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T08:28:01.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:28:01.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-16T08:28:01.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:28:01.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:28:01.540 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T08:28:01.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:28:01.567 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-16T08:28:01.622 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T08:28:01.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:28:01.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T08:28:01.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:28:01.694 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-16T08:28:01.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2147849093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/846172590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:01 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3646852589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T08:28:01.807 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-16T08:28:01.818 INFO:tasks.cephadm:Teardown begin 2024-03-16T08:28:01.818 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T08:28:01.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:28:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2147849093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/846172590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:01 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3646852589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T08:28:01.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 -- 172.21.15.49:0/3050562423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f01005d0 msgr2=0x7f60f01009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 --2- 172.21.15.49:0/3050562423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f01005d0 0x7f60f01009b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ca0 tx=0x7f60e002c0f0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 -- 172.21.15.49:0/3050562423 shutdown_connections 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 --2- 172.21.15.49:0/3050562423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f010e180 0x7f60f0110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 --2- 172.21.15.49:0/3050562423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f0100f80 0x7f60f010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 --2- 172.21.15.49:0/3050562423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f01005d0 0x7f60f01009b0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 -- 172.21.15.49:0/3050562423 >> 172.21.15.49:0/3050562423 conn(0x7f60f00777e0 msgr2=0x7f60f0077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 -- 172.21.15.49:0/3050562423 shutdown_connections 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 -- 172.21.15.49:0/3050562423 wait complete. 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.958+0000 7f60f611b640 1 Processor -- start 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 -- start start 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f01005d0 0x7f60f019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f010e180 0x7f60f019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0113e00 con 0x7f60f0100f80 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0113c80 con 0x7f60f010e180 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60f611b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0113f80 con 0x7f60f01005d0 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33658/0 (socket says 172.21.15.49:33658) 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 -- 172.21.15.49:0/2615074570 learned_addr learned my addr 172.21.15.49:0/2615074570 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60ef7fe640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f01005d0 0x7f60f019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60effff640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f010e180 0x7f60f019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 -- 172.21.15.49:0/2615074570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f01005d0 msgr2=0x7f60f019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f01005d0 0x7f60f019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 -- 172.21.15.49:0/2615074570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f010e180 msgr2=0x7f60f019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f010e180 0x7f60f019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 -- 172.21.15.49:0/2615074570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f019f2b0 con 0x7f60f0100f80 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.959+0000 7f60eeffd640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9f0 tx=0x7f60d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.960+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d800cd80 con 0x7f60f0100f80 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.960+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d8010910 con 0x7f60f0100f80 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.960+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f019f5a0 con 0x7f60f0100f80 2024-03-16T08:28:01.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.960+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8018990 con 0x7f60f0100f80 2024-03-16T08:28:01.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.960+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01a6e30 con 0x7f60f0100f80 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.961+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f00685d0 con 0x7f60f0100f80 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.962+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f60d8010ac0 con 0x7f60f0100f80 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.963+0000 7f60ecff9640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 0x7f60dc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.963+0000 7f60ef7fe640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 0x7f60dc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.963+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7f60d80cf5c0 con 0x7f60f0100f80 2024-03-16T08:28:01.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.963+0000 7f60ef7fe640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 0x7f60dc0a4e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004af0 tx=0x7f60e0005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:01.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:01.966+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f60d809bc90 con 0x7f60f0100f80 2024-03-16T08:28:01.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2147849093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/846172590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T08:28:01.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:01 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3646852589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T08:28:02.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.192+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f019b610 con 0x7f60f0100f80 2024-03-16T08:28:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.193+0000 7f60ecff9640 1 -- 172.21.15.49:0/2615074570 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7f60f019b610 con 0x7f60f0100f80 2024-03-16T08:28:02.193 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:28:02.194 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:28:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 msgr2=0x7f60dc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 0x7f60dc0a4e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004af0 tx=0x7f60e0005c50 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 msgr2=0x7f60f019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9f0 tx=0x7f60d800eeb0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 shutdown_connections 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f60dc0a2970 0x7f60dc0a4e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60f010e180 0x7f60f019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f0100f80 0x7f60f019a800 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 --2- 172.21.15.49:0/2615074570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60f01005d0 0x7f60f019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 >> 172.21.15.49:0/2615074570 conn(0x7f60f00777e0 msgr2=0x7f60f010f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:02.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 shutdown_connections 2024-03-16T08:28:02.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.195+0000 7f60f611b640 1 -- 172.21.15.49:0/2615074570 wait complete. 2024-03-16T08:28:02.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 -- 172.21.15.49:0/4029531271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554101770 msgr2=0x7fc554101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 --2- 172.21.15.49:0/4029531271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554101770 0x7fc554101bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc540009890 tx=0x7fc54002bb70 comp rx=0 tx=0).stop 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 -- 172.21.15.49:0/4029531271 shutdown_connections 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 --2- 172.21.15.49:0/4029531271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554102130 0x7fc55410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 --2- 172.21.15.49:0/4029531271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554101770 0x7fc554101bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 --2- 172.21.15.49:0/4029531271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554107780 0x7fc554107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 -- 172.21.15.49:0/4029531271 >> 172.21.15.49:0/4029531271 conn(0x7fc5540fd280 msgr2=0x7fc5540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 -- 172.21.15.49:0/4029531271 shutdown_connections 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.315+0000 7fc55ac59640 1 -- 172.21.15.49:0/4029531271 wait complete. 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 Processor -- start 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 -- start start 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554102130 0x7fc5541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554107780 0x7fc55419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc554111b90 con 0x7fc554101770 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc554111a10 con 0x7fc554102130 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc55ac59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc554111d10 con 0x7fc554107780 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33678/0 (socket says 172.21.15.49:33678) 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 -- 172.21.15.49:0/2211219879 learned_addr learned my addr 172.21.15.49:0/2211219879 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5591f6640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554107780 0x7fc55419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc553fff640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554102130 0x7fc5541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 -- 172.21.15.49:0/2211219879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554107780 msgr2=0x7fc55419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554107780 0x7fc55419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 -- 172.21.15.49:0/2211219879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554102130 msgr2=0x7fc5541985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554102130 0x7fc5541985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.316+0000 7fc5589f5640 1 -- 172.21.15.49:0/2211219879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55419d0a0 con 0x7fc554101770 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc5589f5640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e9f0 tx=0x7fc54800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54800cd80 con 0x7fc554101770 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55419d390 con 0x7fc554101770 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc548005500 con 0x7fc554101770 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548010690 con 0x7fc554101770 2024-03-16T08:28:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.317+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541a4c20 con 0x7fc554101770 2024-03-16T08:28:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.318+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc554068a20 con 0x7fc554101770 2024-03-16T08:28:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.319+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc5480046c0 con 0x7fc554101770 2024-03-16T08:28:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.320+0000 7fc551ffb640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 0x7fc5440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.320+0000 7fc553fff640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 0x7fc5440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.320+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8492+0+0 (secure 0 0 0) 0x7fc5480cf1d0 con 0x7fc554101770 2024-03-16T08:28:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.320+0000 7fc553fff640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 0x7fc5440a4e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097b0 tx=0x7fc5400058d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.324+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc54809bf50 con 0x7fc554101770 2024-03-16T08:28:02.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.546+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc554199400 con 0x7fc554101770 2024-03-16T08:28:02.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.547+0000 7fc551ffb640 1 -- 172.21.15.49:0/2211219879 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11875 (secure 0 0 0) 0x7fc54809b850 con 0x7fc554101770 2024-03-16T08:28:02.547 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:28:02.547 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2399676198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2399676198},{"type":"v1","addr":"172.21.15.49:6835","nonce":2399676198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/761407043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":761407043},{"type":"v1","addr":"172.21.15.49:6837","nonce":761407043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/2351115602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":2351115602},{"type":"v1","addr":"172.21.15.49:6839","nonce":2351115602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1594522521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1594522521},{"type":"v1","addr":"172.21.15.49:6841","nonce":1594522521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/866170540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":866170540},{"type":"v1","addr":"172.21.15.154:6833","nonce":866170540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3770218709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3770218709},{"type":"v1","addr":"172.21.15.129:6833","nonce":3770218709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T08:15:01.568993+0000","modified":"2024-03-16T08:16:58.308050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24505,"mds_2":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/272945510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":272945510},{"type":"v1","addr":"172.21.15.129:6835","nonce":272945510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2047612746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2047612746},{"type":"v1","addr":"172.21.15.154:6835","nonce":2047612746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/4234449434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4234449434},{"type":"v1","addr":"172.21.15.154:6837","nonce":4234449434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.129:6837/1811728297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1811728297},{"type":"v1","addr":"172.21.15.129:6837","nonce":1811728297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/1693946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":1693946357},{"type":"v1","addr":"172.21.15.154:6839","nonce":1693946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/1288779286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1288779286},{"type":"v1","addr":"172.21.15.129:6839","nonce":1288779286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24505,24491]},"id":1}]} 2024-03-16T08:28:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 msgr2=0x7fc5440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 0x7fc5440a4e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097b0 tx=0x7fc5400058d0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 msgr2=0x7fc5541980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e9f0 tx=0x7fc54800edf0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 shutdown_connections 2024-03-16T08:28:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fc5440a2970 0x7fc5440a4e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc554107780 0x7fc55419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc554102130 0x7fc5541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 --2- 172.21.15.49:0/2211219879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc554101770 0x7fc5541980b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 >> 172.21.15.49:0/2211219879 conn(0x7fc5540fd280 msgr2=0x7fc5540fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 shutdown_connections 2024-03-16T08:28:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.549+0000 7fc55ac59640 1 -- 172.21.15.49:0/2211219879 wait complete. 2024-03-16T08:28:02.561 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:28:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 -- 172.21.15.49:0/2230255682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181006e0 msgr2=0x7fe318100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 --2- 172.21.15.49:0/2230255682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181006e0 0x7fe318100b60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe308009930 tx=0x7fe30802bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 -- 172.21.15.49:0/2230255682 shutdown_connections 2024-03-16T08:28:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 --2- 172.21.15.49:0/2230255682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 0x7fe31810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 --2- 172.21.15.49:0/2230255682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181006e0 0x7fe318100b60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 --2- 172.21.15.49:0/2230255682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181066f0 0x7fe318106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 -- 172.21.15.49:0/2230255682 >> 172.21.15.49:0/2230255682 conn(0x7fe3180777e0 msgr2=0x7fe318077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 -- 172.21.15.49:0/2230255682 shutdown_connections 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 -- 172.21.15.49:0/2230255682 wait complete. 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.678+0000 7fe3206e8640 1 Processor -- start 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 -- start start 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181006e0 0x7fe318102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 0x7fe3181032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe318110b00 con 0x7fe3181066f0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe318110980 con 0x7fe3181006e0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe3206e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe318110c80 con 0x7fe3181010a0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36004/0 (socket says 172.21.15.49:36004) 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 -- 172.21.15.49:0/3524625476 learned_addr learned my addr 172.21.15.49:0/3524625476 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31dc83640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 0x7fe3181032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31e484640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181006e0 0x7fe318102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 -- 172.21.15.49:0/3524625476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 msgr2=0x7fe3181032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 0x7fe3181032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 -- 172.21.15.49:0/3524625476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181006e0 msgr2=0x7fe318102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181006e0 0x7fe318102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 -- 172.21.15.49:0/3524625476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181a65e0 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.679+0000 7fe31ec85640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe31400eb10 tx=0x7fe31400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004b20 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3181a68d0 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe314004cc0 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314010760 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3181a6df0 con 0x7fe3181066f0 2024-03-16T08:28:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.680+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3180685d0 con 0x7fe3181066f0 2024-03-16T08:28:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.682+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fe314005000 con 0x7fe3181066f0 2024-03-16T08:28:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.682+0000 7fe30f7fe640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 0x7fe3040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.682+0000 7fe31e484640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 0x7fe3040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.682+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8536+0+0 (secure 0 0 0) 0x7fe3140cf620 con 0x7fe3181066f0 2024-03-16T08:28:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.683+0000 7fe31e484640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 0x7fe3040a4e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe310009b50 tx=0x7fe310005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:02.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.686+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe314053740 con 0x7fe3181066f0 2024-03-16T08:28:02.860 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:02 smithi129 ceph-mon[36561]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:28:02.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:02 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2615074570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:02.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:02 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2211219879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.873+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe318109c60 con 0x7fe3181066f0 2024-03-16T08:28:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.875+0000 7fe30f7fe640 1 -- 172.21.15.49:0/3524625476 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23122 (secure 0 0 0) 0x7fe314086020 con 0x7fe3181066f0 2024-03-16T08:28:02.875 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-16T08:28:02.875 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":98,"fsid":"053f2308-e36c-11ee-95c9-87774f69a715","created":"2024-03-16T08:07:09.044882+0000","modified":"2024-03-16T08:28:02.593591+0000","last_up_change":"2024-03-16T08:14:12.876932+0000","last_in_change":"2024-03-16T08:13:52.959333+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:10:28.036537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T08:14:59.603963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T08:15:00.940257+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e333c72-baee-40c0-a078-372ada0c1511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6803","nonce":3756414183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6805","nonce":3756414183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6809","nonce":3756414183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3756414183},{"type":"v1","addr":"172.21.15.49:6807","nonce":3756414183}]},"public_addr":"172.21.15.49:6803/3756414183","cluster_addr":"172.21.15.49:6805/3756414183","heartbeat_back_addr":"172.21.15.49:6809/3756414183","heartbeat_front_addr":"172.21.15.49:6807/3756414183","state":["exists","up"]},{"osd":1,"uuid":"79ddf11d-d24d-4221-9317-a912d69ff440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6801","nonce":646273559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6803","nonce":646273559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6807","nonce":646273559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":646273559},{"type":"v1","addr":"172.21.15.129:6805","nonce":646273559}]},"public_addr":"172.21.15.129:6801/646273559","cluster_addr":"172.21.15.129:6803/646273559","heartbeat_back_addr":"172.21.15.129:6807/646273559","heartbeat_front_addr":"172.21.15.129:6805/646273559","state":["exists","up"]},{"osd":2,"uuid":"77bcc416-5d33-4e76-bd0b-7b12aceb1a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6801","nonce":487578356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6803","nonce":487578356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6807","nonce":487578356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":487578356},{"type":"v1","addr":"172.21.15.154:6805","nonce":487578356}]},"public_addr":"172.21.15.154:6801/487578356","cluster_addr":"172.21.15.154:6803/487578356","heartbeat_back_addr":"172.21.15.154:6807/487578356","heartbeat_front_addr":"172.21.15.154:6805/487578356","state":["exists","up"]},{"osd":3,"uuid":"f6b81081-263e-4e72-8499-aea94ed43b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6811","nonce":1033582524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6813","nonce":1033582524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6817","nonce":1033582524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1033582524},{"type":"v1","addr":"172.21.15.49:6815","nonce":1033582524}]},"public_addr":"172.21.15.49:6811/1033582524","cluster_addr":"172.21.15.49:6813/1033582524","heartbeat_back_addr":"172.21.15.49:6817/1033582524","heartbeat_front_addr":"172.21.15.49:6815/1033582524","state":["exists","up"]},{"osd":4,"uuid":"a99b2da5-a3b6-423d-9884-497aae8108de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6809","nonce":615229289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6811","nonce":615229289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6815","nonce":615229289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":615229289},{"type":"v1","addr":"172.21.15.129:6813","nonce":615229289}]},"public_addr":"172.21.15.129:6809/615229289","cluster_addr":"172.21.15.129:6811/615229289","heartbeat_back_addr":"172.21.15.129:6815/615229289","heartbeat_front_addr":"172.21.15.129:6813/615229289","state":["exists","up"]},{"osd":5,"uuid":"9f5563fc-b860-41fd-98a7-5cabd7bb92c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6809","nonce":4188101354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6811","nonce":4188101354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6815","nonce":4188101354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":4188101354},{"type":"v1","addr":"172.21.15.154:6813","nonce":4188101354}]},"public_addr":"172.21.15.154:6809/4188101354","cluster_addr":"172.21.15.154:6811/4188101354","heartbeat_back_addr":"172.21.15.154:6815/4188101354","heartbeat_front_addr":"172.21.15.154:6813/4188101354","state":["exists","up"]},{"osd":6,"uuid":"36672b95-4a0b-4ffc-9103-5f9e2c187b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6819","nonce":3010410053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6821","nonce":3010410053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6825","nonce":3010410053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3010410053},{"type":"v1","addr":"172.21.15.49:6823","nonce":3010410053}]},"public_addr":"172.21.15.49:6819/3010410053","cluster_addr":"172.21.15.49:6821/3010410053","heartbeat_back_addr":"172.21.15.49:6825/3010410053","heartbeat_front_addr":"172.21.15.49:6823/3010410053","state":["exists","up"]},{"osd":7,"uuid":"313db83e-0f1f-4748-b054-77708bd6a27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6817","nonce":252652011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6819","nonce":252652011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6823","nonce":252652011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":252652011},{"type":"v1","addr":"172.21.15.129:6821","nonce":252652011}]},"public_addr":"172.21.15.129:6817/252652011","cluster_addr":"172.21.15.129:6819/252652011","heartbeat_back_addr":"172.21.15.129:6823/252652011","heartbeat_front_addr":"172.21.15.129:6821/252652011","state":["exists","up"]},{"osd":8,"uuid":"c82b22c1-18d5-40dd-8a86-428405308e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6817","nonce":160187571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6819","nonce":160187571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6823","nonce":160187571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":160187571},{"type":"v1","addr":"172.21.15.154:6821","nonce":160187571}]},"public_addr":"172.21.15.154:6817/160187571","cluster_addr":"172.21.15.154:6819/160187571","heartbeat_back_addr":"172.21.15.154:6823/160187571","heartbeat_front_addr":"172.21.15.154:6821/160187571","state":["exists","up"]},{"osd":9,"uuid":"693d0e23-6cdd-41f2-b7c4-f9bb3686edbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6827","nonce":1471236265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6829","nonce":1471236265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6833","nonce":1471236265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1471236265},{"type":"v1","addr":"172.21.15.49:6831","nonce":1471236265}]},"public_addr":"172.21.15.49:6827/1471236265","cluster_addr":"172.21.15.49:6829/1471236265","heartbeat_back_addr":"172.21.15.49:6833/1471236265","heartbeat_front_addr":"172.21.15.49:6831/1471236265","state":["exists","up"]},{"osd":10,"uuid":"7fdf3874-86b2-4175-a20c-41ac7241c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6825","nonce":2725967668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6827","nonce":2725967668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6831","nonce":2725967668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2725967668},{"type":"v1","addr":"172.21.15.129:6829","nonce":2725967668}]},"public_addr":"172.21.15.129:6825/2725967668","cluster_addr":"172.21.15.129:6827/2725967668","heartbeat_back_addr":"172.21.15.129:6831/2725967668","heartbeat_front_addr":"172.21.15.129:6829/2725967668","state":["exists","up"]},{"osd":11,"uuid":"c9663681-0e97-4339-931e-27dc3f68bb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6825","nonce":901406241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6827","nonce":901406241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6831","nonce":901406241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":901406241},{"type":"v1","addr":"172.21.15.154:6829","nonce":901406241}]},"public_addr":"172.21.15.154:6825/901406241","cluster_addr":"172.21.15.154:6827/901406241","heartbeat_back_addr":"172.21.15.154:6831/901406241","heartbeat_front_addr":"172.21.15.154:6829/901406241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:09:37.168928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:00.364752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:24.089419+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:10:49.428272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:13.693349+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:11:38.042466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:03.234762+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:27.835846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:12:53.396848+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:19.443878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:13:44.997642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:14:11.136129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2256169095":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/1561370229":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3786569188":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3066712091":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/372564765":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/1466047614":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/1726594106":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/3762364594":"2024-03-17T08:17:12.732236+0000","172.21.15.49:6800/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:6801/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6800/1374126004":"2024-03-17T08:07:36.305451+0000","172.21.15.49:0/236450725":"2024-03-17T08:17:12.732236+0000","172.21.15.49:0/3199936220":"2024-03-17T08:07:36.305451+0000","172.21.15.49:6801/2659026035":"2024-03-17T08:08:03.498489+0000","172.21.15.49:0/3531600398":"2024-03-17T08:08:03.498489+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:28:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 msgr2=0x7fe3040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 0x7fe3040a4e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe310009b50 tx=0x7fe310005f90 comp rx=0 tx=0).stop 2024-03-16T08:28:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 msgr2=0x7fe318103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe31400eb10 tx=0x7fe31400c2d0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 shutdown_connections 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7fe3040a2970 0x7fe3040a4e30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3181066f0 0x7fe318103820 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3181010a0 0x7fe3181032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 --2- 172.21.15.49:0/3524625476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3181006e0 0x7fe318102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 >> 172.21.15.49:0/3524625476 conn(0x7fe3180777e0 msgr2=0x7fe318068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 shutdown_connections 2024-03-16T08:28:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.877+0000 7fe3206e8640 1 -- 172.21.15.49:0/3524625476 wait complete. 2024-03-16T08:28:02.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:02 smithi049 ceph-mon[31942]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:28:02.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2615074570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:02.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:02 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2211219879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:02.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T08:28:02.975 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:02 smithi154 ceph-mon[36471]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-16T08:28:02.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2615074570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:02.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:02 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2211219879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2757429488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0107790 msgr2=0x7f8ec0107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2757429488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0107790 0x7f8ec0107b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb402bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2757429488 shutdown_connections 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2757429488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0102140 0x7f8ec010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2757429488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2757429488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0107790 0x7f8ec0107b70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2757429488 >> 172.21.15.49:0/2757429488 conn(0x7f8ec00fd210 msgr2=0x7f8ec00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2757429488 shutdown_connections 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2757429488 wait complete. 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 Processor -- start 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:02.999+0000 7f8ec71b2640 1 -- start start 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0102140 0x7f8ec01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0107790 0x7f8ec019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0111b60 con 0x7f8ec0101780 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01119e0 con 0x7f8ec0102140 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0111ce0 con 0x7f8ec0107790 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36014/0 (socket says 172.21.15.49:36014) 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 -- 172.21.15.49:0/2623947781 learned_addr learned my addr 172.21.15.49:0/2623947781 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ebffff640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0102140 0x7f8ec01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec574f640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0107790 0x7f8ec019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 -- 172.21.15.49:0/2623947781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0107790 msgr2=0x7f8ec019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0107790 0x7f8ec019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 -- 172.21.15.49:0/2623947781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0102140 msgr2=0x7f8ec01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0102140 0x7f8ec01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 -- 172.21.15.49:0/2623947781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec019d040 con 0x7f8ec0101780 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec4f4e640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005940 tx=0x7f8eb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4041070 con 0x7f8ec0101780 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec019d2d0 con 0x7f8ec0101780 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb40054e0 con 0x7f8ec0101780 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.000+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4003d00 con 0x7f8ec0101780 2024-03-16T08:28:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.001+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec01a4bc0 con 0x7f8ec0101780 2024-03-16T08:28:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.002+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec0068a20 con 0x7f8ec0101780 2024-03-16T08:28:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.003+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8eb4044070 con 0x7f8ec0101780 2024-03-16T08:28:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.003+0000 7f8ebdffb640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 0x7f8eb00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.003+0000 7f8ebffff640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 0x7f8eb00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.003+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8536+0+0 (secure 0 0 0) 0x7f8eb40f07a0 con 0x7f8ec0101780 2024-03-16T08:28:03.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.004+0000 7f8ebffff640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 0x7f8eb00a4e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0106e80 tx=0x7f8eac005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:03.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.007+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8eb40bcfb0 con 0x7f8ec0101780 2024-03-16T08:28:03.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.231+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8ec0109cf0 con 0x7f8ec0101780 2024-03-16T08:28:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.233+0000 7f8ebdffb640 1 -- 172.21.15.49:0/2623947781 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f8eb40c1050 con 0x7f8ec0101780 2024-03-16T08:28:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T08:28:03.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 msgr2=0x7f8eb00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 0x7f8eb00a4e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0106e80 tx=0x7f8eac005f90 comp rx=0 tx=0).stop 2024-03-16T08:28:03.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 msgr2=0x7f8ec01980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005940 tx=0x7f8eb4004270 comp rx=0 tx=0).stop 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 shutdown_connections 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f8eb00a2970 0x7f8eb00a4e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ec0107790 0x7f8ec019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ec0102140 0x7f8ec01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 --2- 172.21.15.49:0/2623947781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ec0101780 0x7f8ec01980a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 >> 172.21.15.49:0/2623947781 conn(0x7f8ec00fd210 msgr2=0x7f8ec00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 shutdown_connections 2024-03-16T08:28:03.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.236+0000 7f8ec71b2640 1 -- 172.21.15.49:0/2623947781 wait complete. 2024-03-16T08:28:03.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T08:28:03.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 -- 172.21.15.49:0/4146861703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 msgr2=0x7f5bcc100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/4146861703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc100ad0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009930 tx=0x7f5bbc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:03.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 -- 172.21.15.49:0/4146861703 shutdown_connections 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/4146861703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 0x7f5bcc10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/4146861703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc100ad0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/4146861703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 0x7f5bcc106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 -- 172.21.15.49:0/4146861703 >> 172.21.15.49:0/4146861703 conn(0x7f5bcc0777e0 msgr2=0x7f5bcc077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 -- 172.21.15.49:0/4146861703 shutdown_connections 2024-03-16T08:28:03.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 -- 172.21.15.49:0/4146861703 wait complete. 2024-03-16T08:28:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.361+0000 7f5bd1a61640 1 Processor -- start 2024-03-16T08:28:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 -- start start 2024-03-16T08:28:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 0x7f5bcc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 0x7f5bcc1a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc110a70 con 0x7f5bcc100650 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1108f0 con 0x7f5bcc101010 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bd1a61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc110bf0 con 0x7f5bcc106660 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bcaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bcaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36040/0 (socket says 172.21.15.49:36040) 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bcaffd640 1 -- 172.21.15.49:0/271217145 learned_addr learned my addr 172.21.15.49:0/271217145 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bca7fc640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 0x7f5bcc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.362+0000 7f5bcb7fe640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 0x7f5bcc1a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 -- 172.21.15.49:0/271217145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 msgr2=0x7f5bcc1a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 0x7f5bcc1a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 -- 172.21.15.49:0/271217145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 msgr2=0x7f5bcc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 0x7f5bcc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 -- 172.21.15.49:0/271217145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc1a4470 con 0x7f5bcc100650 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bcaffd640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000e9f0 tx=0x7f5bc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc000cd80 con 0x7f5bcc100650 2024-03-16T08:28:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0005500 con 0x7f5bcc100650 2024-03-16T08:28:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1a46a0 con 0x7f5bcc100650 2024-03-16T08:28:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0010690 con 0x7f5bcc100650 2024-03-16T08:28:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.363+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a4bc0 con 0x7f5bcc100650 2024-03-16T08:28:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.364+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc101d90 con 0x7f5bcc100650 2024-03-16T08:28:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.365+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5bc00046c0 con 0x7f5bcc100650 2024-03-16T08:28:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.366+0000 7f5bd0a5f640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 0x7f5bb40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.366+0000 7f5bca7fc640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 0x7f5bb40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.366+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8750+0+0 (secure 0 0 0) 0x7f5bc00cf510 con 0x7f5bcc100650 2024-03-16T08:28:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.366+0000 7f5bca7fc640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 0x7f5bb40a4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005ab0 tx=0x7f5bbc005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.369+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5bc0053740 con 0x7f5bcc100650 2024-03-16T08:28:03.525 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a[31938]: 2024-03-16T08:28:03.231+0000 7fae49a2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:28:03.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:03.598+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5bcc108be0 con 0x7f5bcc100650 2024-03-16T08:28:03.603 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.603 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.603 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mds[67042]: -- 172.21.15.129:0/1979499821 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561497ffda40 con 0x561497fd0800 2024-03-16T08:28:03.603 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: starting mds.b at 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mds[66721]: -- 172.21.15.129:0/4270028024 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56088add1a40 con 0x56088ada5000 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: starting mds.h at 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mds[66414]: -- 172.21.15.129:0/693243554 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ceadd15a40 con 0x55ceadce8800 2024-03-16T08:28:03.604 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: starting mds.e at 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mds[67674]: -- 172.21.15.154:0/1195196503 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8a88b7a40 con 0x55a8a888a800 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: starting mds.c at 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mds[67993]: -- 172.21.15.154:0/1241942251 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560814ac7a40 con 0x560814a9ac00 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: starting mds.i at 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: ignoring --setuser ceph since I am not root 2024-03-16T08:28:03.605 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: ignoring --setgroup ceph since I am not root 2024-03-16T08:28:03.606 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mds[67370]: -- 172.21.15.154:0/1239068659 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f83b841a40 con 0x55f83b814800 2024-03-16T08:28:03.606 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: starting mds.f at 2024-03-16T08:28:03.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/3524625476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T08:28:03.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:03 smithi049 ceph-mon[31942]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/3524625476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T08:28:03.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:03 smithi129 ceph-mon[36561]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/3524625476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/2623947781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T08:28:03.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:03 smithi154 ceph-mon[36471]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T08:28:04.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.234+0000 7f5bd0a5f640 1 -- 172.21.15.49:0/271217145 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f5bc009c130 con 0x7f5bcc100650 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 msgr2=0x7f5bb40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 0x7f5bb40a4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005ab0 tx=0x7f5bbc005fd0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 msgr2=0x7f5bcc198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000e9f0 tx=0x7f5bc000edf0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 shutdown_connections 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f5bb40a29f0 0x7f5bb40a4eb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc106660 0x7f5bcc1a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bcc101010 0x7f5bcc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 --2- 172.21.15.49:0/271217145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bcc100650 0x7f5bcc198eb0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.236+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 >> 172.21.15.49:0/271217145 conn(0x7f5bcc0777e0 msgr2=0x7f5bcc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.237+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 shutdown_connections 2024-03-16T08:28:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.237+0000 7f5bd1a61640 1 -- 172.21.15.49:0/271217145 wait complete. 2024-03-16T08:28:04.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T08:28:04.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 -- 172.21.15.49:0/1448397704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0100660 msgr2=0x7f75d0100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:04.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 --2- 172.21.15.49:0/1448397704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0100660 0x7f75d0100ae0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009930 tx=0x7f75bc02bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:04.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 -- 172.21.15.49:0/1448397704 shutdown_connections 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 --2- 172.21.15.49:0/1448397704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0101020 0x7f75d010d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 --2- 172.21.15.49:0/1448397704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0100660 0x7f75d0100ae0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 --2- 172.21.15.49:0/1448397704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0106670 0x7f75d0106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 -- 172.21.15.49:0/1448397704 >> 172.21.15.49:0/1448397704 conn(0x7f75d0077890 msgr2=0x7f75d0077ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 -- 172.21.15.49:0/1448397704 shutdown_connections 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.362+0000 7f75d6d10640 1 -- 172.21.15.49:0/1448397704 wait complete. 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 Processor -- start 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 -- start start 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0100660 0x7f75d0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0106670 0x7f75d01a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:04.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0110a80 con 0x7f75d0101020 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0110900 con 0x7f75d0100660 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d6d10640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0110c00 con 0x7f75d0106670 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36058/0 (socket says 172.21.15.49:36058) 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75cffff640 1 -- 172.21.15.49:0/566580134 learned_addr learned my addr 172.21.15.49:0/566580134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d4aac640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0100660 0x7f75d0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.363+0000 7f75d52ad640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0106670 0x7f75d01a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 -- 172.21.15.49:0/566580134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0106670 msgr2=0x7f75d01a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0106670 0x7f75d01a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 -- 172.21.15.49:0/566580134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0100660 msgr2=0x7f75d0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0100660 0x7f75d0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 -- 172.21.15.49:0/566580134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a44c0 con 0x7f75d0101020 2024-03-16T08:28:04.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cffff640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009230 tx=0x7f75bc02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc041070 con 0x7f75d0101020 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc004450 con 0x7f75d0101020 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a4690 con 0x7f75d0101020 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc034440 con 0x7f75d0101020 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.364+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a4a90 con 0x7f75d0101020 2024-03-16T08:28:04.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.365+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00685d0 con 0x7f75d0101020 2024-03-16T08:28:04.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.366+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f75bc044070 con 0x7f75d0101020 2024-03-16T08:28:04.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.367+0000 7f75cdffb640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 0x7f75c00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:04.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.367+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8750+0+0 (secure 0 0 0) 0x7f75bc0f0ce0 con 0x7f75d0101020 2024-03-16T08:28:04.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.367+0000 7f75d4aac640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 0x7f75c00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:04.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.368+0000 7f75d4aac640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 0x7f75c00a4e30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f75c400bb60 tx=0x7f75c400b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:04.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.371+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f75bc0bda70 con 0x7f75d0101020 2024-03-16T08:28:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:04.554+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75d0108bd0 con 0x7f75d0101020 2024-03-16T08:28:04.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: Cluster is now healthy 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6836/1169696839,v1:172.21.15.129:6838/1169696839] up:boot 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6837/3328293504,v1:172.21.15.154:6839/3328293504] up:boot 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6834/3044522249,v1:172.21.15.154:6835/3044522249] up:boot 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6834/1642609461,v1:172.21.15.129:6835/1642609461] up:boot 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.129:6837/3057472753,v1:172.21.15.129:6839/3057472753] up:boot 2024-03-16T08:28:04.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: mds.? [v2:172.21.15.154:6836/482070045,v1:172.21.15.154:6838/482070045] up:boot 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: fsmap 12 up:standby 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:28:04.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:04 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: Cluster is now healthy 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6836/1169696839,v1:172.21.15.129:6838/1169696839] up:boot 2024-03-16T08:28:04.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6837/3328293504,v1:172.21.15.154:6839/3328293504] up:boot 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6834/3044522249,v1:172.21.15.154:6835/3044522249] up:boot 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6834/1642609461,v1:172.21.15.129:6835/1642609461] up:boot 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.129:6837/3057472753,v1:172.21.15.129:6839/3057472753] up:boot 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: mds.? [v2:172.21.15.154:6836/482070045,v1:172.21.15.154:6838/482070045] up:boot 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: fsmap 12 up:standby 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:28:04.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:04 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: Cluster is now healthy 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/271217145' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6836/1169696839,v1:172.21.15.129:6838/1169696839] up:boot 2024-03-16T08:28:04.976 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6837/3328293504,v1:172.21.15.154:6839/3328293504] up:boot 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6834/3044522249,v1:172.21.15.154:6835/3044522249] up:boot 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6834/1642609461,v1:172.21.15.129:6835/1642609461] up:boot 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.129:6837/3057472753,v1:172.21.15.129:6839/3057472753] up:boot 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: mds.? [v2:172.21.15.154:6836/482070045,v1:172.21.15.154:6838/482070045] up:boot 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: fsmap 12 up:standby 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='mgr.15432 172.21.15.49:0/2277461506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T08:28:04.977 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:04 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:05.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.236+0000 7f75cdffb640 1 -- 172.21.15.49:0/566580134 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f75bc0bd370 con 0x7f75d0101020 2024-03-16T08:28:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 msgr2=0x7f75c00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 0x7f75c00a4e30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f75c400bb60 tx=0x7f75c400b390 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 msgr2=0x7f75d0199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009230 tx=0x7f75bc02cc70 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 shutdown_connections 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f75c00a2970 0x7f75c00a4e30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d0106670 0x7f75d01a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75d0101020 0x7f75d0199440 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 --2- 172.21.15.49:0/566580134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0100660 0x7f75d0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.239+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 >> 172.21.15.49:0/566580134 conn(0x7f75d0077890 msgr2=0x7f75d0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:05.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.240+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 shutdown_connections 2024-03-16T08:28:05.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.240+0000 7f75d6d10640 1 -- 172.21.15.49:0/566580134 wait complete. 2024-03-16T08:28:05.253 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.370+0000 7f6b5de6c640 1 -- 172.21.15.49:0/3251571266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58102140 msgr2=0x7f6b5810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.370+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/3251571266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58102140 0x7f6b5810e650 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009930 tx=0x7f6b5002bc30 comp rx=0 tx=0).stop 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 -- 172.21.15.49:0/3251571266 shutdown_connections 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/3251571266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58102140 0x7f6b5810e650 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/3251571266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58101780 0x7f6b58101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/3251571266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58107790 0x7f6b58107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 -- 172.21.15.49:0/3251571266 >> 172.21.15.49:0/3251571266 conn(0x7f6b580fd210 msgr2=0x7f6b580ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 -- 172.21.15.49:0/3251571266 shutdown_connections 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 -- 172.21.15.49:0/3251571266 wait complete. 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 Processor -- start 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 -- start start 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.371+0000 7f6b5de6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 0x7f6b58198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 0x7f6b5819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58111ba0 con 0x7f6b58101780 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58111a20 con 0x7f6b58107790 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58111d20 con 0x7f6b58102140 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36068/0 (socket says 172.21.15.49:36068) 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 -- 172.21.15.49:0/1614826659 learned_addr learned my addr 172.21.15.49:0/1614826659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b56ffd640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 0x7f6b58198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b57fff640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 0x7f6b5819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 -- 172.21.15.49:0/1614826659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 msgr2=0x7f6b58198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 0x7f6b58198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 -- 172.21.15.49:0/1614826659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 msgr2=0x7f6b5819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 0x7f6b5819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 -- 172.21.15.49:0/1614826659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5819d060 con 0x7f6b58101780 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b57fff640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 0x7f6b5819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b56ffd640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 0x7f6b58198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b577fe640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800ea80 tx=0x7f6b4800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800ce40 con 0x7f6b58101780 2024-03-16T08:28:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b48005020 con 0x7f6b58101780 2024-03-16T08:28:05.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5819d350 con 0x7f6b58101780 2024-03-16T08:28:05.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48011760 con 0x7f6b58101780 2024-03-16T08:28:05.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.372+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b581a4be0 con 0x7f6b58101780 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.373+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b58068a20 con 0x7f6b58101780 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.374+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f6b480051d0 con 0x7f6b58101780 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.375+0000 7f6b54ff9640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 0x7f6b440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.375+0000 7f6b56ffd640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 0x7f6b440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.375+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8254+0+0 (secure 0 0 0) 0x7f6b480cf3e0 con 0x7f6b58101780 2024-03-16T08:28:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.376+0000 7f6b56ffd640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 0x7f6b440a4e30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b58106e20 tx=0x7f6b40005e50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:28:05.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.380+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b48016030 con 0x7f6b58101780 2024-03-16T08:28:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:05.569+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b581993c0 con 0x7f6b58101780 2024-03-16T08:28:06.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.237+0000 7f6b54ff9640 1 -- 172.21.15.49:0/1614826659 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f6b4809c250 con 0x7f6b58101780 2024-03-16T08:28:06.238 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-03-16T08:28:06.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 msgr2=0x7f6b440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:06.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 0x7f6b440a4e30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b58106e20 tx=0x7f6b40005e50 comp rx=0 tx=0).stop 2024-03-16T08:28:06.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 msgr2=0x7f6b581980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800ea80 tx=0x7f6b4800ef40 comp rx=0 tx=0).stop 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 shutdown_connections 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:6800/4265921640,v1:172.21.15.49:6801/4265921640] conn(0x7f6b440a2970 0x7f6b440a4e30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b58107790 0x7f6b5819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b58102140 0x7f6b58198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 --2- 172.21.15.49:0/1614826659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b58101780 0x7f6b581980c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 >> 172.21.15.49:0/1614826659 conn(0x7f6b580fd210 msgr2=0x7f6b580fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 shutdown_connections 2024-03-16T08:28:06.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-16T08:28:06.241+0000 7f6b5de6c640 1 -- 172.21.15.49:0/1614826659 wait complete. 2024-03-16T08:28:06.257 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:28:06.289 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:28:06.325 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:28:06.360 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-16T08:28:06.360 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T08:28:06.376 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T08:28:06.392 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T08:28:06.417 INFO:tasks.cephadm:Stopping all daemons... 2024-03-16T08:28:06.417 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-16T08:28:06.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a 2024-03-16T08:28:06.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-mon[31942]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-03-16T08:28:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T08:28:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-mon[31942]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T08:28:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-mon[31942]: from='client.? 172.21.15.49:0/1614826659' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:06.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 systemd[1]: Stopping Ceph mon.a for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:06 smithi129 ceph-mon[36561]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-03-16T08:28:06.610 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T08:28:06.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:06 smithi129 ceph-mon[36561]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T08:28:06.611 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:06 smithi129 ceph-mon[36561]: from='client.? 172.21.15.49:0/1614826659' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:06 smithi154 ceph-mon[36471]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-03-16T08:28:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/566580134' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T08:28:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:06 smithi154 ceph-mon[36471]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T08:28:06.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:06 smithi154 ceph-mon[36471]: from='client.? 172.21.15.49:0/1614826659' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T08:28:07.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a[31938]: 2024-03-16T08:28:06.741+0000 7fae4f235700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:07.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a[31938]: 2024-03-16T08:28:06.741+0000 7fae4f235700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-16T08:28:07.276 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:07 smithi049 podman[99327]: 2024-03-16 08:28:07.007740757 +0000 UTC m=+0.376468495 container died 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T08:28:07.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:07 smithi049 podman[99327]: 2024-03-16 08:28:07.367363087 +0000 UTC m=+0.736090826 container cleanup 54b77ba4c5c9866881cea92b5a26262fd60586eaebb617f9826948f12e927d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T08:28:07.538 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 16 08:28:07 smithi049 bash[99327]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-a 2024-03-16T08:28:07.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.a.service' 2024-03-16T08:28:07.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:07.850 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-16T08:28:07.851 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-16T08:28:07.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.b 2024-03-16T08:28:08.169 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:07 smithi129 systemd[1]: Stopping Ceph mon.b for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:08.511 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b[36556]: 2024-03-16T08:28:08.167+0000 7f2adbf7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:08.512 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b[36556]: 2024-03-16T08:28:08.167+0000 7f2adbf7f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-16T08:28:08.512 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 podman[72342]: 2024-03-16 08:28:08.209274006 +0000 UTC m=+0.165048922 container died 1775c1589efda2ea9ba81361b2ef63fd7066deb07ee046b4fc732b6e7796e10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:08.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 podman[72342]: 2024-03-16 08:28:08.602069832 +0000 UTC m=+0.557844742 container cleanup 1775c1589efda2ea9ba81361b2ef63fd7066deb07ee046b4fc732b6e7796e10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T08:28:08.861 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 bash[72342]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b 2024-03-16T08:28:09.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.b.service' 2024-03-16T08:28:09.094 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:08 smithi129 podman[72356]: 2024-03-16 08:28:08.928627396 +0000 UTC m=+0.716465296 container remove 1775c1589efda2ea9ba81361b2ef63fd7066deb07ee046b4fc732b6e7796e10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:28:09.095 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:09 smithi129 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-16T08:28:09.095 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:09 smithi129 systemd[1]: Stopped Ceph mon.b for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:28:09.095 INFO:journalctl@ceph.mon.b.smithi129.stdout:Mar 16 08:28:09 smithi129 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.b.service: Consumed 11.136s CPU time. 2024-03-16T08:28:09.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:09.124 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-16T08:28:09.124 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-16T08:28:09.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.c 2024-03-16T08:28:09.453 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 systemd[1]: Stopping Ceph mon.c for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:09.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c[36467]: 2024-03-16T08:28:09.452+0000 7f082afbd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:09.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c[36467]: 2024-03-16T08:28:09.452+0000 7f082afbd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-16T08:28:09.726 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 podman[73507]: 2024-03-16 08:28:09.473228288 +0000 UTC m=+0.157030334 container died 28848fa0265bda66917a5d516db7b7b6c4c718a066aa3f11596df563bcaf5c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T08:28:10.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 podman[73507]: 2024-03-16 08:28:09.93706401 +0000 UTC m=+0.620866059 container cleanup 28848fa0265bda66917a5d516db7b7b6c4c718a066aa3f11596df563bcaf5c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T08:28:10.226 INFO:journalctl@ceph.mon.c.smithi154.stdout:Mar 16 08:28:09 smithi154 bash[73507]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mon-c 2024-03-16T08:28:10.369 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mon.c.service' 2024-03-16T08:28:10.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:10.429 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-16T08:28:10.429 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-16T08:28:10.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x 2024-03-16T08:28:10.764 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:10 smithi049 systemd[1]: Stopping Ceph mgr.x for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:11.038 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:28:10.763+0000 7fa8cbc23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:11.038 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x[32175]: 2024-03-16T08:28:10.763+0000 7fa8cbc23700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-16T08:28:11.038 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:10 smithi049 podman[99448]: 2024-03-16 08:28:10.795260173 +0000 UTC m=+0.160820395 container died 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-16T08:28:11.433 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:11 smithi049 podman[99448]: 2024-03-16 08:28:11.156275997 +0000 UTC m=+0.521836214 container cleanup 3273f86bb2380cca03fbfd6bd72298aefadb0d66bb6eb90c4b63370802edb12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-16T08:28:11.433 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 16 08:28:11 smithi049 bash[99448]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-x 2024-03-16T08:28:11.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-16T08:28:11.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:11.624 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-16T08:28:11.624 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-16T08:28:11.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.y 2024-03-16T08:28:11.961 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:28:11 smithi129 systemd[1]: Stopping Ceph mgr.y for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:12.265 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:28:11 smithi129 podman[72463]: 2024-03-16 08:28:11.960625351 +0000 UTC m=+0.137437771 container died 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T08:28:12.611 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:28:12 smithi129 podman[72463]: 2024-03-16 08:28:12.374023459 +0000 UTC m=+0.550835875 container cleanup 6c7b1da62a51d243217af07e50b06606cb0e2335a0f09c1d09ae805fbad85306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T08:28:12.611 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Mar 16 08:28:12 smithi129 bash[72463]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-y 2024-03-16T08:28:12.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-16T08:28:12.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:12.829 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-16T08:28:12.829 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-16T08:28:12.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.z 2024-03-16T08:28:13.226 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:28:12 smithi154 systemd[1]: Stopping Ceph mgr.z for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:13.671 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:28:13 smithi154 podman[73627]: 2024-03-16 08:28:13.347088006 +0000 UTC m=+0.328658228 container died 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:28:13.976 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:28:13 smithi154 podman[73627]: 2024-03-16 08:28:13.762314629 +0000 UTC m=+0.743884857 container cleanup 0305239ca55d71ccc7f6ace3360bf33dd6251f6ec5c6ee345560d9e8433f0fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-16T08:28:13.976 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Mar 16 08:28:13 smithi154 bash[73627]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mgr-z 2024-03-16T08:28:14.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-16T08:28:14.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:14.166 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-16T08:28:14.166 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-16T08:28:14.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.0 2024-03-16T08:28:14.287 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:14 smithi049 systemd[1]: Stopping Ceph osd.0 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:14.787 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0[46677]: 2024-03-16T08:28:14.486+0000 7fa65e558700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:14.788 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0[46677]: 2024-03-16T08:28:14.486+0000 7fa65e558700 -1 osd.0 101 *** Got signal Terminated *** 2024-03-16T08:28:14.788 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0[46677]: 2024-03-16T08:28:14.486+0000 7fa65e558700 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:20.121 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:19 smithi049 podman[99568]: 2024-03-16 08:28:19.830061898 +0000 UTC m=+5.455986026 container died bbd2dcf586dfc6b7f69bddf6f31c2ad58b2a6a0b7953e3bf4ab3ed00518234fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T08:28:20.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:20 smithi049 podman[99568]: 2024-03-16 08:28:20.228519728 +0000 UTC m=+5.854443887 container cleanup bbd2dcf586dfc6b7f69bddf6f31c2ad58b2a6a0b7953e3bf4ab3ed00518234fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T08:28:20.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:20 smithi049 bash[99568]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0 2024-03-16T08:28:20.772 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:20 smithi049 podman[99588]: 2024-03-16 08:28:20.455049461 +0000 UTC m=+0.622098133 container remove bbd2dcf586dfc6b7f69bddf6f31c2ad58b2a6a0b7953e3bf4ab3ed00518234fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T08:28:21.037 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:20 smithi049 podman[99677]: 2024-03-16 08:28:20.675981386 +0000 UTC m=+0.018446599 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:28:21.038 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:20 smithi049 podman[99677]: 2024-03-16 08:28:20.814525229 +0000 UTC m=+0.156990421 container create b466cdaeb72243672dfbfd1fb269328cb4598f10066623be76f682775be17f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-16T08:28:21.424 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:21 smithi049 podman[99677]: 2024-03-16 08:28:21.117414277 +0000 UTC m=+0.459879471 container init b466cdaeb72243672dfbfd1fb269328cb4598f10066623be76f682775be17f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T08:28:21.424 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:21 smithi049 podman[99677]: 2024-03-16 08:28:21.120836284 +0000 UTC m=+0.463301486 container start b466cdaeb72243672dfbfd1fb269328cb4598f10066623be76f682775be17f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-16T08:28:21.424 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:21 smithi049 podman[99677]: 2024-03-16 08:28:21.209151127 +0000 UTC m=+0.551616333 container attach b466cdaeb72243672dfbfd1fb269328cb4598f10066623be76f682775be17f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:28:21.780 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 16 08:28:21 smithi049 podman[99677]: 2024-03-16 08:28:21.424342391 +0000 UTC m=+0.766807582 container died b466cdaeb72243672dfbfd1fb269328cb4598f10066623be76f682775be17f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-16T08:28:22.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.0.service' 2024-03-16T08:28:22.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:22.107 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-16T08:28:22.107 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-16T08:28:22.107 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.1 2024-03-16T08:28:22.404 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:22 smithi129 systemd[1]: Stopping Ceph osd.1 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:22.861 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:28:22.419+0000 7f9495a99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:22.861 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:28:22.419+0000 7f9495a99700 -1 osd.1 101 *** Got signal Terminated *** 2024-03-16T08:28:22.861 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1[41174]: 2024-03-16T08:28:22.419+0000 7f9495a99700 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:28.250 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:27 smithi129 podman[72589]: 2024-03-16 08:28:27.951972089 +0000 UTC m=+5.642183041 container died ff5b34a763e7ea2a87fade17c38b6e44dc890b3485a49ef5e299aa872296e469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T08:28:28.597 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:28 smithi129 podman[72589]: 2024-03-16 08:28:28.354135342 +0000 UTC m=+6.044346310 container cleanup ff5b34a763e7ea2a87fade17c38b6e44dc890b3485a49ef5e299aa872296e469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:28.598 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:28 smithi129 bash[72589]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1 2024-03-16T08:28:28.860 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:28 smithi129 podman[72617]: 2024-03-16 08:28:28.597212988 +0000 UTC m=+0.640142829 container remove ff5b34a763e7ea2a87fade17c38b6e44dc890b3485a49ef5e299aa872296e469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:29.218 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:28 smithi129 podman[72707]: 2024-03-16 08:28:28.818577899 +0000 UTC m=+0.017859478 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:28:29.218 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:28 smithi129 podman[72707]: 2024-03-16 08:28:28.923588268 +0000 UTC m=+0.122869898 container create c0ad0503a0c7b25ffe359c0c3a7fb6c27f990907eb732a8864de05aa80e35104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-16T08:28:29.218 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:29 smithi129 podman[72707]: 2024-03-16 08:28:29.159349711 +0000 UTC m=+0.358631281 container init c0ad0503a0c7b25ffe359c0c3a7fb6c27f990907eb732a8864de05aa80e35104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-16T08:28:29.218 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:29 smithi129 podman[72707]: 2024-03-16 08:28:29.162476454 +0000 UTC m=+0.361758020 container start c0ad0503a0c7b25ffe359c0c3a7fb6c27f990907eb732a8864de05aa80e35104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T08:28:29.218 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:29 smithi129 podman[72707]: 2024-03-16 08:28:29.217665027 +0000 UTC m=+0.416946608 container attach c0ad0503a0c7b25ffe359c0c3a7fb6c27f990907eb732a8864de05aa80e35104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T08:28:29.611 INFO:journalctl@ceph.osd.1.smithi129.stdout:Mar 16 08:28:29 smithi129 podman[72707]: 2024-03-16 08:28:29.449985006 +0000 UTC m=+0.649266574 container died c0ad0503a0c7b25ffe359c0c3a7fb6c27f990907eb732a8864de05aa80e35104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T08:28:30.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.1.service' 2024-03-16T08:28:30.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:30.250 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-16T08:28:30.250 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-16T08:28:30.251 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.2 2024-03-16T08:28:30.609 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:30 smithi154 systemd[1]: Stopping Ceph osd.2 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:30.976 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2[41621]: 2024-03-16T08:28:30.607+0000 7fc5366f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:30.976 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2[41621]: 2024-03-16T08:28:30.607+0000 7fc5366f4700 -1 osd.2 101 *** Got signal Terminated *** 2024-03-16T08:28:30.976 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2[41621]: 2024-03-16T08:28:30.607+0000 7fc5366f4700 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:36.226 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:35 smithi154 podman[73763]: 2024-03-16 08:28:35.935199869 +0000 UTC m=+5.492455487 container died 0b60d8089ffe7bbae41ef3db8442af04144a7df52f9e6d5d1328516eb4c09093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T08:28:36.693 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:36 smithi154 podman[73763]: 2024-03-16 08:28:36.391619987 +0000 UTC m=+5.948875611 container cleanup 0b60d8089ffe7bbae41ef3db8442af04144a7df52f9e6d5d1328516eb4c09093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T08:28:36.693 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:36 smithi154 bash[73763]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2 2024-03-16T08:28:36.976 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:36 smithi154 podman[73780]: 2024-03-16 08:28:36.693194335 +0000 UTC m=+0.748785236 container remove 0b60d8089ffe7bbae41ef3db8442af04144a7df52f9e6d5d1328516eb4c09093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T08:28:37.306 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:36.910702884 +0000 UTC m=+0.017650169 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:28:37.307 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:37.019177441 +0000 UTC m=+0.126124755 container create 7f4ddcbfbe6527a9a24a5fb50bd257cbb579b126ba671173915a5269e6f1d732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-16T08:28:37.307 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:37.305941801 +0000 UTC m=+0.412889093 container init 7f4ddcbfbe6527a9a24a5fb50bd257cbb579b126ba671173915a5269e6f1d732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T08:28:37.604 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:37.309685441 +0000 UTC m=+0.416632718 container start 7f4ddcbfbe6527a9a24a5fb50bd257cbb579b126ba671173915a5269e6f1d732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:28:37.604 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:37.398940341 +0000 UTC m=+0.505887633 container attach 7f4ddcbfbe6527a9a24a5fb50bd257cbb579b126ba671173915a5269e6f1d732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:28:37.969 INFO:journalctl@ceph.osd.2.smithi154.stdout:Mar 16 08:28:37 smithi154 podman[73875]: 2024-03-16 08:28:37.603587093 +0000 UTC m=+0.710534388 container died 7f4ddcbfbe6527a9a24a5fb50bd257cbb579b126ba671173915a5269e6f1d732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-16T08:28:38.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.2.service' 2024-03-16T08:28:38.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:38.344 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-16T08:28:38.345 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-16T08:28:38.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.3 2024-03-16T08:28:38.666 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:38 smithi049 systemd[1]: Stopping Ceph osd.3 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:39.037 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:38 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3[53378]: 2024-03-16T08:28:38.664+0000 7f1bb0a5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:39.038 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:38 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3[53378]: 2024-03-16T08:28:38.664+0000 7f1bb0a5e700 -1 osd.3 101 *** Got signal Terminated *** 2024-03-16T08:28:39.038 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:38 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3[53378]: 2024-03-16T08:28:38.664+0000 7f1bb0a5e700 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:41.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:40.728+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:42.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:41.733+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:42.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:42.712+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:43.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:42 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:42.769+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:43.476 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:43.217+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:43.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:43.672+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:44.113 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:43 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:43.804+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:44.113 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:43 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:43.834+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:44.364 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:44 smithi049 podman[99786]: 2024-03-16 08:28:44.113098968 +0000 UTC m=+5.560743405 container died 5f09a2c080c850213ae9afcfc4470e7f9ef8f8a7c0081ceab5ae7f5a40bfd01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T08:28:44.673 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:44.254+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:44.673 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:44.461+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:44.673 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:44 smithi049 podman[99786]: 2024-03-16 08:28:44.454972187 +0000 UTC m=+5.902616624 container cleanup 5f09a2c080c850213ae9afcfc4470e7f9ef8f8a7c0081ceab5ae7f5a40bfd01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:28:44.674 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:44 smithi049 bash[99786]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3 2024-03-16T08:28:44.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:44.672+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:45.037 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:44 smithi049 podman[99815]: 2024-03-16 08:28:44.673206061 +0000 UTC m=+0.553287121 container remove 5f09a2c080c850213ae9afcfc4470e7f9ef8f8a7c0081ceab5ae7f5a40bfd01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:45.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:44 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:44.764+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:45.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:44 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:44.875+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:45.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:44 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:44.776+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:45.344 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:44.988603352 +0000 UTC m=+0.019157803 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:28:45.344 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:45.107554966 +0000 UTC m=+0.138109417 container create a3ea15aaa26f247b0c3243c33a2c5ea1e8d47ffe4acdc7d206feb5e595cf97aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T08:28:45.344 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:45.285474061 +0000 UTC m=+0.316028528 container init a3ea15aaa26f247b0c3243c33a2c5ea1e8d47ffe4acdc7d206feb5e595cf97aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:45.344 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:45.289240921 +0000 UTC m=+0.319795372 container start a3ea15aaa26f247b0c3243c33a2c5ea1e8d47ffe4acdc7d206feb5e595cf97aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T08:28:45.344 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:45.343660351 +0000 UTC m=+0.374214814 container attach a3ea15aaa26f247b0c3243c33a2c5ea1e8d47ffe4acdc7d206feb5e595cf97aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-16T08:28:45.682 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:45.431+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:45.682 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:45.294+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:45.906 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 16 08:28:45 smithi049 podman[99904]: 2024-03-16 08:28:45.596858806 +0000 UTC m=+0.627413248 container died a3ea15aaa26f247b0c3243c33a2c5ea1e8d47ffe4acdc7d206feb5e595cf97aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:28:45.906 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:45 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:45.752+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:45.943 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:45.681+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:46.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:45 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:45.765+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:46.181 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:45 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:45.918+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:46.208 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.3.service' 2024-03-16T08:28:46.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:46.257 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-16T08:28:46.257 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-16T08:28:46.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.4 2024-03-16T08:28:46.575 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:46 smithi129 systemd[1]: Stopping Ceph osd.4 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:46.715 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:46.304+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:46.716 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:46.395+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:46.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:46 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:46.766+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:46.861 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:46 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:46.573+0000 7f00aca57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:46.861 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:46 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:46.573+0000 7f00aca57700 -1 osd.4 101 *** Got signal Terminated *** 2024-03-16T08:28:46.861 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:46 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:46.573+0000 7f00aca57700 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:46.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:46.715+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:47.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:46 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:46.775+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:47.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:46 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:46.877+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:47.694 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:47 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:47.346+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:47.695 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:47 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:47.434+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:47.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:47 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:47.693+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:48.094 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:47 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:47.762+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:48.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:47 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:47.811+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:48.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:47 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:47.901+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:48.655 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:48 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:48.322+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:48.655 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:48 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:48.474+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:48.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:48 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:48.654+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:48.994 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:48 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:48.734+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:49.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:48 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:48.761+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:49.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:48 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:48.761+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:49.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:48 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:48.893+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:49.360 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:48 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:48.993+0000 7f00a5043700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.583788+0000 front 2024-03-16T08:28:24.583766+0000 (oldest deadline 2024-03-16T08:28:48.083964+0000) 2024-03-16T08:28:49.698 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:49 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:49.301+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:49.698 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:49 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:49.466+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:49.698 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:49 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:49.466+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:49.774 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:49 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:49.491+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:49.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:49 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:49.697+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:50.025 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:49 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:49.773+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:50.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:49.804+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:50.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:49.804+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:50.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:49.875+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:50.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:49.875+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:50.360 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:50 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:50.024+0000 7f00a5043700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.583788+0000 front 2024-03-16T08:28:24.583766+0000 (oldest deadline 2024-03-16T08:28:48.083964+0000) 2024-03-16T08:28:50.724 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:50 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:50.306+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:50.724 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:50 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:50.460+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:50.724 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:50 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:50.460+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:50.810 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:50 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:50.493+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:50.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:50 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:50.723+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:50.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:50 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:50.723+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:51.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:50 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:50.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:51.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:50 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:50.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:51.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:50.799+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:51.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:50.799+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:51.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:50.896+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:51.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:50 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:50.896+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:51.360 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:51 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4[47871]: 2024-03-16T08:28:51.072+0000 7f00a5043700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.583788+0000 front 2024-03-16T08:28:24.583766+0000 (oldest deadline 2024-03-16T08:28:48.083964+0000) 2024-03-16T08:28:51.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:51 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:51.311+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:51.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:51 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:51.506+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:51.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:51 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:51.506+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:51.857 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:51 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:51.446+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:52.111 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:52 smithi129 podman[72813]: 2024-03-16 08:28:52.086163911 +0000 UTC m=+5.642188925 container died b7b86ab39a2cb0dd1f3e94bb82e1d06bfe351022cf5ba415c60b3ec3c9efb670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:28:52.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:51 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:51.855+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:52.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:51 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:51.855+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:52.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:51 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:51.728+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:52.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:51 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:51.728+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:52.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:51 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:51.798+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:52.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:51 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:51.798+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:52.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:51 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:51.898+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:52.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:51 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:51.898+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:52.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:52 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:52.326+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:52.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:52 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:52.467+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:52.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:52 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:52.467+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:52.758 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:52 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:52.493+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:52.759 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:52 smithi129 podman[72813]: 2024-03-16 08:28:52.474293398 +0000 UTC m=+6.030318403 container cleanup b7b86ab39a2cb0dd1f3e94bb82e1d06bfe351022cf5ba415c60b3ec3c9efb670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T08:28:52.759 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:52 smithi129 bash[72813]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4 2024-03-16T08:28:53.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:52 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:52.761+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:53.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:52 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:52.761+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:53.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:52 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:52.880+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:53.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:52 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:52.880+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:53.086 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:52 smithi129 podman[72830]: 2024-03-16 08:28:52.758769694 +0000 UTC m=+0.671779181 container remove b7b86ab39a2cb0dd1f3e94bb82e1d06bfe351022cf5ba415c60b3ec3c9efb670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T08:28:53.086 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:52 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:52.808+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:53.086 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:52 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:52.808+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:53.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:52 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:52.761+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:53.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:52 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:52.761+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:53.360 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:52.988441275 +0000 UTC m=+0.017908302 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:28:53.361 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:53.109607868 +0000 UTC m=+0.139074870 container create b87311a04c8a006075171271434ff3ce8074dd021feea7123e9fc861dc4d0321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:53.361 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:53.296123939 +0000 UTC m=+0.325590962 container init b87311a04c8a006075171271434ff3ce8074dd021feea7123e9fc861dc4d0321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:28:53.361 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:53.299502367 +0000 UTC m=+0.328969376 container start b87311a04c8a006075171271434ff3ce8074dd021feea7123e9fc861dc4d0321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-16T08:28:53.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:53.511+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:53.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:53.511+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:53.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:53.360+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:53.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:53.360+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:53.811 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:53.387301068 +0000 UTC m=+0.416768092 container attach b87311a04c8a006075171271434ff3ce8074dd021feea7123e9fc861dc4d0321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T08:28:53.811 INFO:journalctl@ceph.osd.4.smithi129.stdout:Mar 16 08:28:53 smithi129 podman[72919]: 2024-03-16 08:28:53.586923315 +0000 UTC m=+0.616390338 container died b87311a04c8a006075171271434ff3ce8074dd021feea7123e9fc861dc4d0321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T08:28:53.812 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:53 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:53.464+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:54.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:53 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:53.762+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:54.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:53 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:53.762+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:54.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:53 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:53.857+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:54.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:53 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:53.857+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:54.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:53 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:53.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:54.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:53 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:53.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:54.160 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.4.service' 2024-03-16T08:28:54.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:28:54.204 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-16T08:28:54.204 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-16T08:28:54.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.5 2024-03-16T08:28:54.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:53.777+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:54.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:53 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:53.777+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:54.509 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 systemd[1]: Stopping Ceph osd.5 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:28:54.509 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:54.313+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:54.509 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:54.313+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:54.827 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:54.507+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:54.827 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:54.507+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:54.827 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:54.530+0000 7f1ebe9b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:28:54.827 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:54.530+0000 7f1ebe9b2700 -1 osd.5 101 *** Got signal Terminated *** 2024-03-16T08:28:54.827 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:54.530+0000 7f1ebe9b2700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:28:54.835 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:54 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:54.448+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:55.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:54.749+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:55.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:54.749+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:55.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:54.817+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:55.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:54.817+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:55.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:54 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:54.835+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:55.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:54 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:54.835+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:55.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:54.826+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:55.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:54 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:54.826+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:55.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:55.339+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:55.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:55.339+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:55.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:55.478+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:55.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:55.478+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:55.811 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:55 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:55.409+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:56.037 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:55 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:55.733+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:56.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:55 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:55.733+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:56.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:55 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:55.830+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:56.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:55 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:55.830+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:56.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:55 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:55.811+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:56.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:55 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:55.811+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:56.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:55.777+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:56.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:55 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:55.777+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:56.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:56.500+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:56.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:56.500+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:56.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:56.360+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:56.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:56.360+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:56.843 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:56 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:56.455+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:57.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:56.757+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:57.038 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:56.757+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:57.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:56.866+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:57.038 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:56 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:56.866+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:57.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:56 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:56.843+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:57.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:56 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:56.843+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:57.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:56.820+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:57.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:56 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:56.820+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:57.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:57.484+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:57.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:57.484+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:57.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:57.392+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:57.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:57.392+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:57.845 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:57 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:57.451+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:58.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:57 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:57.844+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:58.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:57 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:57.844+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:58.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:57.799+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:58.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:57 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:57.799+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:58.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:57.798+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:58.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:57.798+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:58.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:57.864+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:58.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:57 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:57.864+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:58.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:58.483+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:58.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:58.483+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:58.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:58.441+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:58.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:58.441+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:58.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:58.441+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:28:58.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:58 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:58.472+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:28:59.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:58.774+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:28:59.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:58 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:58.774+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:28:59.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:58.808+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:28:59.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:58.808+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:28:59.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:58.913+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:28:59.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:58.913+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:28:59.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:58 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:58.913+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:28:59.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:58 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:58.885+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:28:59.361 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:58 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:58.885+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:28:59.361 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:58 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:58.885+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:59.476+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.147807+0000 front 2024-03-16T08:28:19.147912+0000 (oldest deadline 2024-03-16T08:28:44.447810+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:59.476+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.448454+0000 front 2024-03-16T08:28:24.448558+0000 (oldest deadline 2024-03-16T08:28:48.548426+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5[48415]: 2024-03-16T08:28:59.476+0000 7f1ebafa6700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.549958+0000 front 2024-03-16T08:28:35.550078+0000 (oldest deadline 2024-03-16T08:28:59.050016+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:59.458+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:59.458+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:28:59.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:28:59.458+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:28:59.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:28:59 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:28:59.438+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:00.150 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:28:59 smithi154 podman[73996]: 2024-03-16 08:28:59.841108191 +0000 UTC m=+5.447138977 container died 38f31445e02d43d5ddee48a553caee7e26ae54c42f68df89d9329b0aee5d7a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:29:00.150 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:59.726+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:00.150 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:59.726+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:00.150 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:28:59 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:28:59.726+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:59.847+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:59.847+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:28:59.847+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:59.910+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:59.910+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:00.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:28:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:28:59.910+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:00.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:59 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:59.899+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:00.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:59 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:59.899+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:00.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:28:59 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:28:59.899+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:00.456 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:00 smithi154 podman[73996]: 2024-03-16 08:29:00.249308183 +0000 UTC m=+5.855338961 container cleanup 38f31445e02d43d5ddee48a553caee7e26ae54c42f68df89d9329b0aee5d7a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-16T08:29:00.457 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:00 smithi154 bash[73996]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5 2024-03-16T08:29:00.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:00 smithi154 podman[74014]: 2024-03-16 08:29:00.484013916 +0000 UTC m=+0.639706960 container remove 38f31445e02d43d5ddee48a553caee7e26ae54c42f68df89d9329b0aee5d7a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T08:29:00.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:00.456+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:00.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:00.456+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:00.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:00.456+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:00.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:00 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:00.456+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:01.055 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:00 smithi154 podman[74108]: 2024-03-16 08:29:00.819431621 +0000 UTC m=+0.017638245 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:01.055 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:00 smithi154 podman[74108]: 2024-03-16 08:29:00.943184129 +0000 UTC m=+0.141390746 container create 0da51a162de57be6ae746f749048701cdb524520583409665b9eaaa1d5eb6a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T08:29:01.055 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:00.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:01.055 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:00.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:01.055 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:00 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:00.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:01.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:00.833+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:01.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:00.833+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:01.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:00.833+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:01.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:00.944+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:01.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:00.944+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:01.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:00 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:00.944+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:01.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:00 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:00.921+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:01.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:00 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:00.921+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:01.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:00 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:00.921+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:01.452 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:01 smithi154 podman[74108]: 2024-03-16 08:29:01.171867691 +0000 UTC m=+0.370074307 container init 0da51a162de57be6ae746f749048701cdb524520583409665b9eaaa1d5eb6a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-16T08:29:01.452 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:01 smithi154 podman[74108]: 2024-03-16 08:29:01.175199057 +0000 UTC m=+0.373405662 container start 0da51a162de57be6ae746f749048701cdb524520583409665b9eaaa1d5eb6a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T08:29:01.452 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:01 smithi154 podman[74108]: 2024-03-16 08:29:01.271233545 +0000 UTC m=+0.469440170 container attach 0da51a162de57be6ae746f749048701cdb524520583409665b9eaaa1d5eb6a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:29:01.725 INFO:journalctl@ceph.osd.5.smithi154.stdout:Mar 16 08:29:01 smithi154 podman[74108]: 2024-03-16 08:29:01.463361083 +0000 UTC m=+0.661567693 container died 0da51a162de57be6ae746f749048701cdb524520583409665b9eaaa1d5eb6a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:29:01.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:01.450+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:01.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:01.450+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:01.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:01.450+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:01.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:01.498+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:01.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:01.498+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:02.133 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:01.743+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:02.133 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:01.743+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:02.133 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:01 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:01.743+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:02.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.5.service' 2024-03-16T08:29:02.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:02.243 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-16T08:29:02.243 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-16T08:29:02.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.6 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:01.928+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:01.928+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:01.928+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:01.848+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:01.848+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:02.277 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:01 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:01.848+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:02.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:01.873+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:02.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:01.873+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:02.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:01 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:01.873+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:02.537 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 systemd[1]: Stopping Ceph osd.6 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:02.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:02.474+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:02.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:02.474+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:02.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:02.474+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:02.839 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:02 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:02.453+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:02.839 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:02 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:02.453+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:02.842 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.549+0000 7fdffbdee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:02.842 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.549+0000 7fdffbdee700 -1 osd.6 101 *** Got signal Terminated *** 2024-03-16T08:29:02.842 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.549+0000 7fdffbdee700 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:02.842 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.841+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:02.843 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.841+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:02.843 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:02.841+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:03.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:02 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:02.838+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:03.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:02 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:02.838+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:03.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:02 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:02.838+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:03.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:02.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:03.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:02.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:03.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:02 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:02.731+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:03.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:02.907+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:03.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:02.907+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:03.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:02 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:02.907+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:03.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:03.442+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:03.860 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:03.442+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:03.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:03.717+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:03.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:03.717+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:03.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:03.717+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:03.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:03.492+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:03.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:03.492+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:03.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:03 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:03.492+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:04.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:03.867+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:04.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:03.867+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:04.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:03.867+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:04.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:03.924+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:04.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:03.924+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:04.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:03 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:03.924+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:04.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:03.860+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:04.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:03.860+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:04.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:03 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:03.860+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:04.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:04.460+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:04.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:04.460+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:04.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:04.460+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:04.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:04.396+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:04.816 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:04.396+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:04.816 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:04.396+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:05.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:04.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:05.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:04.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:05.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:04.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:05.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:04 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:04.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:05.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:04.733+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:05.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:04.733+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:05.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:04.733+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:05.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:04 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:04.733+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:04.882+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:04.882+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:04.882+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:04.868+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:04.868+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:04.868+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:05.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:04.868+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.702014+0000 front 2024-03-16T08:28:39.702084+0000 (oldest deadline 2024-03-16T08:29:04.402198+0000) 2024-03-16T08:29:05.704 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:05.419+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:05.704 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:05.419+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:05.704 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:05.419+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:05.835 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:05.370+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:05.836 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:05.370+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:05.836 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:05.370+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:05.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:05.703+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:05.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:05.703+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:05.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:05.703+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:05.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:05 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:05.703+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:06.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:05.834+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:06.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:05.834+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:06.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:05.834+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:06.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:05 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:05.834+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:06.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:05.878+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:05.878+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:05.878+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:05.877+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:05.877+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:05.877+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:06.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:05 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:05.877+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.702014+0000 front 2024-03-16T08:28:39.702084+0000 (oldest deadline 2024-03-16T08:29:04.402198+0000) 2024-03-16T08:29:06.701 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:06.443+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:06.701 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:06.443+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:06.701 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:06.443+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:06.810 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:06.382+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:06.810 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:06.382+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:06.810 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:06.382+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:06.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:06.701+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:06.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:06.701+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:06.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:06.701+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:06.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:06 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:06.701+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:07.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:06.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:07.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:06.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:07.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:06.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:07.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:06 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:06.809+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:07.287 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:06.855+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.999062+0000 front 2024-03-16T08:28:17.999105+0000 (oldest deadline 2024-03-16T08:28:42.099195+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:06.855+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:22.099750+0000 front 2024-03-16T08:28:22.099833+0000 (oldest deadline 2024-03-16T08:28:47.999890+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:06.855+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.801628+0000 front 2024-03-16T08:28:33.801545+0000 (oldest deadline 2024-03-16T08:28:59.701640+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6[59889]: 2024-03-16T08:29:06.855+0000 7fdff83e2700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.702014+0000 front 2024-03-16T08:28:39.702084+0000 (oldest deadline 2024-03-16T08:29:04.402198+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:06.915+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:06.915+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:06.915+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:07.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:06 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:06.915+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:07.708 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:07.457+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:07.708 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:07.457+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:07.708 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:07.457+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:07.806 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:07.411+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:07.806 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:07.411+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:07.806 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:07.411+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:07.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:07.708+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:07.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:07.708+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:07.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:07.708+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:07.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:07 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:07.708+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:08.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:07.805+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:08.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:07.805+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:08.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:07.805+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:08.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:07 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:07.805+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:08.161 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:07 smithi049 podman[100009]: 2024-03-16 08:29:07.896348903 +0000 UTC m=+5.444779020 container died 64ef359ebc5f61f9763f31781d85cf2028e533a255059c7fa21d466d0730916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:29:08.161 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:07.911+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:08.161 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:07.911+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:08.161 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:07.911+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:08.161 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:07 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:07.911+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:08.501 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:08 smithi049 podman[100009]: 2024-03-16 08:29:08.266784428 +0000 UTC m=+5.815214517 container cleanup 64ef359ebc5f61f9763f31781d85cf2028e533a255059c7fa21d466d0730916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-03-16T08:29:08.501 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:08 smithi049 bash[100009]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6 2024-03-16T08:29:08.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:08.466+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:08.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:08.466+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:08.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:08.466+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:08.726 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:08.664+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:08.726 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:08.664+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:08.726 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:08.664+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:08.726 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:08 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:08.664+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:08.788 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:08 smithi049 podman[100027]: 2024-03-16 08:29:08.501334732 +0000 UTC m=+0.608156854 container remove 64ef359ebc5f61f9763f31781d85cf2028e533a255059c7fa21d466d0730916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T08:29:08.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:08.402+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:08.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:08.402+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:08.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:08.402+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:09.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:08.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:09.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:08.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:09.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:08.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:09.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:08.815+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:08 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:08.957+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:08 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:08.957+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:08 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:08.957+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:08 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:08.957+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:08 smithi049 podman[100120]: 2024-03-16 08:29:08.742027471 +0000 UTC m=+0.017754717 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:08 smithi049 podman[100120]: 2024-03-16 08:29:08.843859021 +0000 UTC m=+0.119586259 container create 22cbf0c0e9219651a29128d12e4b6bf8c1391e44b004ad5252b9f00ece137a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:09 smithi049 podman[100120]: 2024-03-16 08:29:09.06319295 +0000 UTC m=+0.338920189 container init 22cbf0c0e9219651a29128d12e4b6bf8c1391e44b004ad5252b9f00ece137a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-16T08:29:09.155 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:09 smithi049 podman[100120]: 2024-03-16 08:29:09.066297271 +0000 UTC m=+0.342024513 container start 22cbf0c0e9219651a29128d12e4b6bf8c1391e44b004ad5252b9f00ece137a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-16T08:29:09.156 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:09 smithi049 podman[100120]: 2024-03-16 08:29:09.154904589 +0000 UTC m=+0.430631844 container attach 22cbf0c0e9219651a29128d12e4b6bf8c1391e44b004ad5252b9f00ece137a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T08:29:09.538 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 16 08:29:09 smithi049 podman[100120]: 2024-03-16 08:29:09.363899667 +0000 UTC m=+0.639626921 container died 22cbf0c0e9219651a29128d12e4b6bf8c1391e44b004ad5252b9f00ece137a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T08:29:09.802 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:09.364+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:09.802 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:09.364+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:09.802 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:09.364+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:09.956 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.6.service' 2024-03-16T08:29:09.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:09.478+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:09.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:09.478+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:09.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:09.478+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:09.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:09.676+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:09.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:09.676+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:09.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:09.676+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:09.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:09 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:09.676+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:10.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:10.044 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-16T08:29:10.044 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-16T08:29:10.045 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.7 2024-03-16T08:29:10.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:09.802+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:10.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:09.802+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:10.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:09.802+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:10.073 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:09 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:09.802+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:10.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.005+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:10.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.005+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:10.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.005+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:10.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.005+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:10.344 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 systemd[1]: Stopping Ceph osd.7 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:10.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.343+0000 7fae7e0f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:10.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.343+0000 7fae7e0f9700 -1 osd.7 101 *** Got signal Terminated *** 2024-03-16T08:29:10.611 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.343+0000 7fae7e0f9700 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:10.611 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.398+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:10.611 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.398+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:10.611 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:10.398+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:10.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:10.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:10.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:10.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:10.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:10.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:10.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:10.687+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:10.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:10.687+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:10.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:10.687+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:10.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:10 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:10.687+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:11.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:10.776+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:11.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:10.776+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:11.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:10.776+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:11.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:10 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:10.776+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:11.271 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.988+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:11.271 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.988+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:11.271 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.988+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:11.271 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:10 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:10.988+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:11.751 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:11.363+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:11.751 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:11.363+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:11.751 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:11.363+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:11.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:11.552+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:11.552+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:11.552+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:11.689+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:11.689+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:11.689+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:11.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:11 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:11.689+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:12.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:11.750+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:12.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:11.750+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:12.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:11.750+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:12.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:11 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:11.750+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:12.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:12.038+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:12.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:12.038+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:12.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:12.038+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:12.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:12 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:12.038+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:12.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:12.358+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:12.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:12.358+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:12.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:12.358+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:12.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:12.549+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:12.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:12.549+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:12.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:12.549+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:12.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:12.727+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:12.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:12.727+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:12.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:12.727+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:12.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:12 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:12.727+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:13.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:12.774+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:13.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:12.774+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:13.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:12.774+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:13.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:12.774+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:13.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:13 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:13.025+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:13.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:13 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:13.025+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:13.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:13 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:13.025+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:13.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:13 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:13.025+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:13.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:13.326+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:13.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:13.326+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:13.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:13.326+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:13.586+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:13.586+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:13.586+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:13.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:13.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:13.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:13.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:13 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:13.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:14.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:13.795+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:14.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:13.795+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:14.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:13.795+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:14.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:13 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:13.795+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:14.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:14.057+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:14.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:14.057+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:14.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:14.057+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:14.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:14 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:14.057+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:14.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:14.329+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.070865+0000 front 2024-03-16T08:28:27.070935+0000 (oldest deadline 2024-03-16T08:28:49.371015+0000) 2024-03-16T08:29:14.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:14.329+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.572448+0000 front 2024-03-16T08:28:34.572563+0000 (oldest deadline 2024-03-16T08:29:00.472459+0000) 2024-03-16T08:29:14.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7[54552]: 2024-03-16T08:29:14.329+0000 7fae7a6ed700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:40.973565+0000 front 2024-03-16T08:28:40.973546+0000 (oldest deadline 2024-03-16T08:29:04.373950+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:14.616+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:14.616+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:14.616+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:14.749+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:14.749+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:14.749+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:14.877 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:14 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:14.749+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:15.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:14.763+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:15.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:14.763+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:15.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:14.763+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:15.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:14 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:14.763+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:15.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:15 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:15.055+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:15.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:15 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:15.055+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:15.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:15 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:15.055+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:15.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:15 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:15.055+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:15.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:15.619+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:15.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:15.619+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:15.890 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:15.619+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:15.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:15.785+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:15.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:15.785+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:15.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:15.785+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:15.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:15 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:15.785+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:15.994 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:15 smithi129 podman[73025]: 2024-03-16 08:29:15.727998664 +0000 UTC m=+5.488787234 container died ab3a1cb9909a42c99f084812b0e63d4e15c746a574fba6a674572fdb306d20d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:29:15.995 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:15 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:15.737+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:15.995 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:15 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:15.737+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:15.995 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:15 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:15.737+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:15.995 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:15 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:15.737+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:16.336 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73025]: 2024-03-16 08:29:16.077010492 +0000 UTC m=+5.837799038 container cleanup ab3a1cb9909a42c99f084812b0e63d4e15c746a574fba6a674572fdb306d20d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:16.336 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 bash[73025]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7 2024-03-16T08:29:16.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:16.081+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:16.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:16.081+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:16.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:16.081+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:16.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:16 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:16.081+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:16.610 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73042]: 2024-03-16 08:29:16.336380137 +0000 UTC m=+0.605120601 container remove ab3a1cb9909a42c99f084812b0e63d4e15c746a574fba6a674572fdb306d20d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:16.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:16.751+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:16.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:16.751+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:16.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:16.751+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:16.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:16.751+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:16.977 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:16.580+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:16.977 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:16.580+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:16.977 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:16 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:16.580+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:16.744+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:16.744+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:16.744+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:16.744+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:16.744+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73133]: 2024-03-16 08:29:16.581391794 +0000 UTC m=+0.017742722 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73133]: 2024-03-16 08:29:16.695649671 +0000 UTC m=+0.132000617 container create b626b8a18c1a1b07ae30664f533a724796d16ef97ea792b9a90ef73496af848c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73133]: 2024-03-16 08:29:16.898143745 +0000 UTC m=+0.334494666 container init b626b8a18c1a1b07ae30664f533a724796d16ef97ea792b9a90ef73496af848c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73133]: 2024-03-16 08:29:16.90149358 +0000 UTC m=+0.337844483 container start b626b8a18c1a1b07ae30664f533a724796d16ef97ea792b9a90ef73496af848c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:29:16.981 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:16 smithi129 podman[73133]: 2024-03-16 08:29:16.981125369 +0000 UTC m=+0.417476284 container attach b626b8a18c1a1b07ae30664f533a724796d16ef97ea792b9a90ef73496af848c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-16T08:29:17.360 INFO:journalctl@ceph.osd.7.smithi129.stdout:Mar 16 08:29:17 smithi129 podman[73133]: 2024-03-16 08:29:17.189868893 +0000 UTC m=+0.626219797 container died b626b8a18c1a1b07ae30664f533a724796d16ef97ea792b9a90ef73496af848c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:17.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:17.068+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:17.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:17.068+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:17.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:17.068+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:17.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:17 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:17.068+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:17.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.7.service' 2024-03-16T08:29:17.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:17.926 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-16T08:29:17.927 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-16T08:29:17.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.8 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:17.594+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:17.594+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:17.594+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:17.594+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:17.762+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:17.762+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:17.762+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:17.955 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:17 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:17.762+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:18.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:17.779+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:18.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:17.779+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:18.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:17.779+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:18.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:17.779+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:18.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:17 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:17.779+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:18.218 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:17 smithi154 systemd[1]: Stopping Ceph osd.8 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:18.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:18.028+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:18.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:18.028+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:18.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:18.028+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:18.288 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:18 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:18.028+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:18.475 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.218+0000 7f7d46b0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:18.475 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.218+0000 7f7d46b0b700 -1 osd.8 101 *** Got signal Terminated *** 2024-03-16T08:29:18.475 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.218+0000 7f7d46b0b700 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:18.975 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.758+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.758+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.758+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:18.758+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:18.585+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:18.585+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:18.585+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:18.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:18 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:18.585+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:19.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:18.735+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:19.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:18.735+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:19.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:18.735+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:19.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:18.735+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:19.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:18 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:18.735+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:19.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:19.077+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:19.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:19.077+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:19.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:19.077+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:19.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:19 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:19.077+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:19.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:19.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:19.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:19.771+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:19.537+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:19.537+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:19.537+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:19.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:19 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:19.537+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:20.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:19.731+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:20.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:19.731+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:20.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:19.731+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:20.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:19.731+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:20.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:19 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:19.731+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:20.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:20.121+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:20.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:20.121+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:20.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:20.121+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:20.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:20 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:20.121+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:20.776 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:20.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:20.776 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:20.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:20.776 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:20.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:20.776 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:20.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:21.012 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:20.725+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:21.012 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:20.725+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:21.012 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:20.725+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:21.012 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:20.725+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:21.012 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:20.725+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:21.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:20.774+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:21.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:20.774+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:21.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:20.775+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:21.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:20.775+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:21.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:20 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:20.775+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:57.654358+0000 front 2024-03-16T08:28:57.654324+0000 (oldest deadline 2024-03-16T08:29:20.554412+0000) 2024-03-16T08:29:21.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:21.139+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:21.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:21.139+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:21.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:21.139+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:21.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:21.139+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:21.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:21 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:21.139+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:21.792 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:21.514+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:21.792 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:21.514+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:21.792 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:21.514+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:21.792 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:21.514+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:21.792 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:21.514+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:22.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:22.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:22.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:22.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:22.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:22.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:21 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:21.726+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:22.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:21.791+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:22.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:21.791+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:22.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:21.791+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:22.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:21.791+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:22.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:21 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:21.791+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:57.654358+0000 front 2024-03-16T08:28:57.654324+0000 (oldest deadline 2024-03-16T08:29:20.554412+0000) 2024-03-16T08:29:22.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:22.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:22.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:22.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:22.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:22.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:22.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:22.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:22.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:22 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:22.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:22.831 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:22.510+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:22.831 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:22.510+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:22.832 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:22.510+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:22.832 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:22.510+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:22.832 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:22.510+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:23.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:23.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:23.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:23.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:23.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:23.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:22 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:22.706+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:23.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:22.831+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:13.947731+0000 front 2024-03-16T08:28:13.947666+0000 (oldest deadline 2024-03-16T08:28:39.847741+0000) 2024-03-16T08:29:23.225 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:22.831+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.649527+0000 front 2024-03-16T08:28:25.649636+0000 (oldest deadline 2024-03-16T08:28:50.349680+0000) 2024-03-16T08:29:23.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:22.831+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.850559+0000 front 2024-03-16T08:28:33.850474+0000 (oldest deadline 2024-03-16T08:28:59.150610+0000) 2024-03-16T08:29:23.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:22.831+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.451657+0000 front 2024-03-16T08:28:41.451742+0000 (oldest deadline 2024-03-16T08:29:04.351427+0000) 2024-03-16T08:29:23.226 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:22 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8[55407]: 2024-03-16T08:29:22.831+0000 7f7d3f0f7700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:57.654358+0000 front 2024-03-16T08:28:57.654324+0000 (oldest deadline 2024-03-16T08:29:20.554412+0000) 2024-03-16T08:29:23.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:23.217+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:23.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:23.217+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:23.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:23.217+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:23.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:23.217+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:23.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:23 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:23.217+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:23.819 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:23 smithi154 podman[74226]: 2024-03-16 08:29:23.539001355 +0000 UTC m=+5.423762885 container died cbc213c89528491491ed8902d923d41fc203e31b89f14cd0409f56bbecf82570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-16T08:29:23.820 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:23 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:23.536+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:23.820 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:23 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:23.536+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:23.820 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:23 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:23.536+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:23.820 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:23 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:23.536+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:23.820 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:23 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:23.536+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:24.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:24.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:24.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:24.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:24.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:24.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:23 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:23.680+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:24.195 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:23 smithi154 podman[74226]: 2024-03-16 08:29:23.910400768 +0000 UTC m=+5.795162299 container cleanup cbc213c89528491491ed8902d923d41fc203e31b89f14cd0409f56bbecf82570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-16T08:29:24.195 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:23 smithi154 bash[74226]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8 2024-03-16T08:29:24.475 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74243]: 2024-03-16 08:29:24.195163823 +0000 UTC m=+0.651328040 container remove cbc213c89528491491ed8902d923d41fc203e31b89f14cd0409f56bbecf82570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-03-16T08:29:24.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:24 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:24.179+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:24.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:24 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:24.179+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:24.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:24 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:24.179+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:24.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:24 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:24.179+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:24.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:24 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:24.179+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:24.815 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:24 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:24.520+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:24.815 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:24 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:24.521+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:24.815 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:24 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:24.521+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:24.815 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:24 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:24.521+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:24.815 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:24 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:24.521+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:24.816 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74331]: 2024-03-16 08:29:24.42127986 +0000 UTC m=+0.017761100 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:24.816 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74331]: 2024-03-16 08:29:24.546011976 +0000 UTC m=+0.142493245 container create 5a24bb1224bbce64739c9675483a392a9aa3e3020a92acb6bf5d5ed6235b9df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T08:29:24.816 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74331]: 2024-03-16 08:29:24.81519152 +0000 UTC m=+0.411672756 container init 5a24bb1224bbce64739c9675483a392a9aa3e3020a92acb6bf5d5ed6235b9df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T08:29:25.107 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74331]: 2024-03-16 08:29:24.818698827 +0000 UTC m=+0.415180046 container start 5a24bb1224bbce64739c9675483a392a9aa3e3020a92acb6bf5d5ed6235b9df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T08:29:25.107 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:24 smithi154 podman[74331]: 2024-03-16 08:29:24.898338403 +0000 UTC m=+0.494819640 container attach 5a24bb1224bbce64739c9675483a392a9aa3e3020a92acb6bf5d5ed6235b9df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:25.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:25.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:25.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:25.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:25.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:25.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:24 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:24.636+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:25.401 INFO:journalctl@ceph.osd.8.smithi154.stdout:Mar 16 08:29:25 smithi154 podman[74331]: 2024-03-16 08:29:25.107518548 +0000 UTC m=+0.703999784 container died 5a24bb1224bbce64739c9675483a392a9aa3e3020a92acb6bf5d5ed6235b9df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T08:29:25.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:25.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:25.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:25.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:25.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:25.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:25.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:25.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:25.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:25.185+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:25.685 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:25.507+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:25.685 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:25.507+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:25.685 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:25.507+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:25.685 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:25.507+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:25.685 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:25.507+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:25.716 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.8.service' 2024-03-16T08:29:25.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:25.801 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-16T08:29:25.801 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-16T08:29:25.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.9 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:25.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:25 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:25.604+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:26.128 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:25 smithi049 systemd[1]: Stopping Ceph osd.9 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.127+0000 7f15cf308700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.127+0000 7f15cf308700 -1 osd.9 101 *** Got signal Terminated *** 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.127+0000 7f15cf308700 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.166+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.166+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.166+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.166+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:26.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:26 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:26.166+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:26.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:26 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:26.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:26.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:26 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:26.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:26.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:26 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:26.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:26.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:26 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:26.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:26.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:26 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:26.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:27.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:27.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:27.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:27.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:27.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:27.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:26 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:26.642+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:27.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:27 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:27.142+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:27.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:27 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:27.142+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:27.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:27 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:27.142+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:27.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:27 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:27.142+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:27.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:27 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:27.142+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:27.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:27 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:27.517+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:27.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:27 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:27.517+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:27.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:27 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:27.517+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:27.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:27 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:27.517+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:27.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:27 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:27.517+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:28.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:28.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:28.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:28.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:28.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:28.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:27 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:27.629+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:28.537 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:28 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:28.118+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:28.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:28 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:28.118+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:28.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:28 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:28.118+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:28.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:28 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:28.118+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:28.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:28 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:28.118+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:28.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:28.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:28.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:28.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:28.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:28.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:28.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:28.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:28.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:28.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:29.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:29.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:28 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:28.650+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:29.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:29.168+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:29.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:29.168+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:29.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:29.168+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:29.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:29.168+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:29.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:29 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:29.168+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:29.860 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:29.860 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:29.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:29.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:29.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:29.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:29.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:29 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:29.605+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:29.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:29 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:29.498+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:29.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:29 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:29.498+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:29.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:29 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:29.498+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:29.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:29 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:29.498+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:29.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:29 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:29.498+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:30.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:30.208+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.494735+0000 front 2024-03-16T08:28:17.494682+0000 (oldest deadline 2024-03-16T08:28:43.394864+0000) 2024-03-16T08:29:30.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:30.208+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:25.096098+0000 front 2024-03-16T08:28:25.096163+0000 (oldest deadline 2024-03-16T08:28:49.196160+0000) 2024-03-16T08:29:30.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:30.208+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:33.797874+0000 front 2024-03-16T08:28:33.797824+0000 (oldest deadline 2024-03-16T08:28:58.497830+0000) 2024-03-16T08:29:30.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:30.208+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:41.998586+0000 front 2024-03-16T08:28:41.998798+0000 (oldest deadline 2024-03-16T08:29:06.698855+0000) 2024-03-16T08:29:30.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:30 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9[66777]: 2024-03-16T08:29:30.208+0000 7f15cb8fc700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:59.501693+0000 front 2024-03-16T08:28:59.501729+0000 (oldest deadline 2024-03-16T08:29:20.601554+0000) 2024-03-16T08:29:30.860 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:30.861 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:30 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:30.608+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:30.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:30.531+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:30.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:30.531+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:30.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:30.531+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:30.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:30.531+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:30.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:30 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:30.531+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:31.918 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:31 smithi049 podman[100237]: 2024-03-16 08:29:31.603762757 +0000 UTC m=+5.599148348 container died 259d62a13e60f2ab00294fccf8438d8922e3dab578d4ca19b4a6de82db73cf79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-16T08:29:31.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:31 smithi049 podman[100237]: 2024-03-16 08:29:31.918678926 +0000 UTC m=+5.914064523 container cleanup 259d62a13e60f2ab00294fccf8438d8922e3dab578d4ca19b4a6de82db73cf79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-16T08:29:31.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:31.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:31.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:31.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:31.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:31.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:31 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:31.578+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:32.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:32.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:31 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:31.627+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:32.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:31 smithi049 bash[100237]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9 2024-03-16T08:29:32.287 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100251]: 2024-03-16 08:29:32.094887109 +0000 UTC m=+0.486512313 container remove 259d62a13e60f2ab00294fccf8438d8922e3dab578d4ca19b4a6de82db73cf79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:32.823 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100344]: 2024-03-16 08:29:32.545598935 +0000 UTC m=+0.114027487 container create 170a7b1869746a30cb15eaf2d5d134b3052032195ad9d614b49cbb6a6e383510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T08:29:32.823 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100344]: 2024-03-16 08:29:32.450317641 +0000 UTC m=+0.018746200 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:32.823 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100344]: 2024-03-16 08:29:32.748382819 +0000 UTC m=+0.316811384 container init 170a7b1869746a30cb15eaf2d5d134b3052032195ad9d614b49cbb6a6e383510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T08:29:32.823 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100344]: 2024-03-16 08:29:32.751849275 +0000 UTC m=+0.320277834 container start 170a7b1869746a30cb15eaf2d5d134b3052032195ad9d614b49cbb6a6e383510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-16T08:29:32.823 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:32 smithi049 podman[100344]: 2024-03-16 08:29:32.822868231 +0000 UTC m=+0.391296792 container attach 170a7b1869746a30cb15eaf2d5d134b3052032195ad9d614b49cbb6a6e383510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T08:29:32.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:32.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:32.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:32.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:32.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:32.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:32.625+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:33.087 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 16 08:29:33 smithi049 podman[100344]: 2024-03-16 08:29:33.059060308 +0000 UTC m=+0.627488871 container died 170a7b1869746a30cb15eaf2d5d134b3052032195ad9d614b49cbb6a6e383510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T08:29:33.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:33.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:33.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:33.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:33.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:33.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:33.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:32 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:32.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:33.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.9.service' 2024-03-16T08:29:33.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:33.661 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-16T08:29:33.661 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-16T08:29:33.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.10 2024-03-16T08:29:33.971 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:33.971 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:33.971 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:33.971 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:33.971 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.640+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 systemd[1]: Stopping Ceph osd.10 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.969+0000 7f4ba1aea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.969+0000 7f4ba1aea700 -1 osd.10 101 *** Got signal Terminated *** 2024-03-16T08:29:33.972 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:33 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:33.969+0000 7f4ba1aea700 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:33.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:33.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:33.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:33.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:33.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:33.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:33 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:33.603+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:34.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:34.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:34.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:34.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:34.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:34.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:34 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:34.572+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:35.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:35.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:35.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:35.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:35.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:35.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:35.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:34 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:34.652+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:35.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:35.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:35.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:35.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:35.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:35.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:35 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:35.596+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:36.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:36.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:35 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:35.675+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:36.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:36.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:36.551+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:37.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:37.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:36 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:36.655+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:37.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:37.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:37 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:37.542+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:38.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:38.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:38.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:38.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:38.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:38.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:38.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:37 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:37.626+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:38.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:38.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:38 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:38.524+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:39.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:19.324988+0000 front 2024-03-16T08:28:19.325027+0000 (oldest deadline 2024-03-16T08:28:44.624988+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:24.625372+0000 front 2024-03-16T08:28:24.625336+0000 (oldest deadline 2024-03-16T08:28:49.925536+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:35.027714+0000 front 2024-03-16T08:28:35.027735+0000 (oldest deadline 2024-03-16T08:28:57.927583+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-16T08:28:39.028468+0000 front 2024-03-16T08:28:39.028439+0000 (oldest deadline 2024-03-16T08:29:04.328439+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.030429+0000 front 2024-03-16T08:28:50.030383+0000 (oldest deadline 2024-03-16T08:29:15.930503+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:55.931052+0000 front 2024-03-16T08:28:55.930893+0000 (oldest deadline 2024-03-16T08:29:21.230933+0000) 2024-03-16T08:29:39.111 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:38 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10[61617]: 2024-03-16T08:29:38.670+0000 7f4b9e0de700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.131979+0000 front 2024-03-16T08:29:07.132043+0000 (oldest deadline 2024-03-16T08:29:28.132340+0000) 2024-03-16T08:29:39.704 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:39 smithi129 podman[73250]: 2024-03-16 08:29:39.439437278 +0000 UTC m=+5.587684084 container died 58d3a2679a72b67cc8357dcdb5843f45fb13980ab86245b4592d4e515638bec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:39.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:39.977 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:39 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:39.516+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:40.037 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:39 smithi129 podman[73250]: 2024-03-16 08:29:39.803207954 +0000 UTC m=+5.951454751 container cleanup 58d3a2679a72b67cc8357dcdb5843f45fb13980ab86245b4592d4e515638bec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T08:29:40.051 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:39 smithi129 bash[73250]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10 2024-03-16T08:29:40.360 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73268]: 2024-03-16 08:29:40.037537833 +0000 UTC m=+0.596630428 container remove 58d3a2679a72b67cc8357dcdb5843f45fb13980ab86245b4592d4e515638bec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T08:29:40.674 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.263877429 +0000 UTC m=+0.017706238 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:40.674 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.375466729 +0000 UTC m=+0.129295531 container create 9a5e5cba9669289f4efae1f791f4bb86cc32827132d30cde16337553f3da6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307) 2024-03-16T08:29:40.674 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.599990192 +0000 UTC m=+0.353819007 container init 9a5e5cba9669289f4efae1f791f4bb86cc32827132d30cde16337553f3da6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-16T08:29:40.674 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.603014901 +0000 UTC m=+0.356843696 container start 9a5e5cba9669289f4efae1f791f4bb86cc32827132d30cde16337553f3da6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:29:40.674 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.673868687 +0000 UTC m=+0.427697495 container attach 9a5e5cba9669289f4efae1f791f4bb86cc32827132d30cde16337553f3da6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T08:29:40.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:40.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:40 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:40.509+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:41.110 INFO:journalctl@ceph.osd.10.smithi129.stdout:Mar 16 08:29:40 smithi129 podman[73361]: 2024-03-16 08:29:40.892311832 +0000 UTC m=+0.646140626 container died 9a5e5cba9669289f4efae1f791f4bb86cc32827132d30cde16337553f3da6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:29:41.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.10.service' 2024-03-16T08:29:41.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:41.681 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-16T08:29:41.681 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-16T08:29:41.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.11 2024-03-16T08:29:41.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.491+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:41.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 systemd[1]: Stopping Ceph osd.11 for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:42.475 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.995+0000 7f244bb8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:42.475 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.995+0000 7f244bb8c700 -1 osd.11 101 *** Got signal Terminated *** 2024-03-16T08:29:42.475 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:41 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:41.995+0000 7f244bb8c700 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T08:29:42.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:42.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:42 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:42.488+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:43.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:43.976 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:43 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:43.535+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:44.877 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:44 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:44.486+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:45.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:45.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:45 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:45.449+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-03-16T08:29:21.864759+0000 front 2024-03-16T08:29:21.864796+0000 (oldest deadline 2024-03-16T08:29:44.764826+0000) 2024-03-16T08:29:46.725 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-16T08:28:17.754662+0000 front 2024-03-16T08:28:17.754775+0000 (oldest deadline 2024-03-16T08:28:42.454753+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-03-16T08:28:27.155977+0000 front 2024-03-16T08:28:27.155897+0000 (oldest deadline 2024-03-16T08:28:52.455812+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-03-16T08:28:34.156547+0000 front 2024-03-16T08:28:34.156512+0000 (oldest deadline 2024-03-16T08:28:57.656748+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-03-16T08:28:50.960197+0000 front 2024-03-16T08:28:50.960263+0000 (oldest deadline 2024-03-16T08:29:16.760811+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-03-16T08:28:56.761276+0000 front 2024-03-16T08:28:56.761295+0000 (oldest deadline 2024-03-16T08:29:21.461419+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-16T08:29:07.362491+0000 front 2024-03-16T08:29:07.362398+0000 (oldest deadline 2024-03-16T08:29:30.862304+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-03-16T08:29:11.963669+0000 front 2024-03-16T08:29:11.963755+0000 (oldest deadline 2024-03-16T08:29:36.063571+0000) 2024-03-16T08:29:46.726 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:46 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11[62478]: 2024-03-16T08:29:46.429+0000 7f2444178700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-03-16T08:29:21.864759+0000 front 2024-03-16T08:29:21.864796+0000 (oldest deadline 2024-03-16T08:29:44.764826+0000) 2024-03-16T08:29:47.579 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:47 smithi154 podman[74436]: 2024-03-16 08:29:47.311055985 +0000 UTC m=+5.435373317 container died 9c5849242124d342ff687e4829751b318bb43f225047c3add32929ab4490cdd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-16T08:29:47.954 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:47 smithi154 podman[74436]: 2024-03-16 08:29:47.678340558 +0000 UTC m=+5.802657856 container cleanup 9c5849242124d342ff687e4829751b318bb43f225047c3add32929ab4490cdd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-16T08:29:47.954 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:47 smithi154 bash[74436]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11 2024-03-16T08:29:48.225 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:47 smithi154 podman[74466]: 2024-03-16 08:29:47.954454551 +0000 UTC m=+0.639860581 container remove 9c5849242124d342ff687e4829751b318bb43f225047c3add32929ab4490cdd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:29:48.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.280478209 +0000 UTC m=+0.107996288 container create e1eba04a89f4c0f67a90effe51ed6a144b5babc27abf1340d3e42d22b148839b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:48.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.189974126 +0000 UTC m=+0.017492201 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T08:29:48.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.491546028 +0000 UTC m=+0.319064110 container init e1eba04a89f4c0f67a90effe51ed6a144b5babc27abf1340d3e42d22b148839b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T08:29:48.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.494874145 +0000 UTC m=+0.322392217 container start e1eba04a89f4c0f67a90effe51ed6a144b5babc27abf1340d3e42d22b148839b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:48.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.56610886 +0000 UTC m=+0.393626955 container attach e1eba04a89f4c0f67a90effe51ed6a144b5babc27abf1340d3e42d22b148839b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:48.975 INFO:journalctl@ceph.osd.11.smithi154.stdout:Mar 16 08:29:48 smithi154 podman[74557]: 2024-03-16 08:29:48.782612068 +0000 UTC m=+0.610130149 container died e1eba04a89f4c0f67a90effe51ed6a144b5babc27abf1340d3e42d22b148839b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:49.352 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@osd.11.service' 2024-03-16T08:29:49.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:49.393 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-16T08:29:49.393 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-16T08:29:49.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.a 2024-03-16T08:29:49.696 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:49 smithi049 systemd[1]: Stopping Ceph mds.a for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:50.038 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a[73343]: 2024-03-16T08:29:49.694+0000 7f8ea673b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:50.038 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:49 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a[73343]: 2024-03-16T08:29:49.694+0000 7f8ea673b700 -1 mds.a *** got signal Terminated *** 2024-03-16T08:29:53.962 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:53 smithi049 podman[100466]: 2024-03-16 08:29:53.708104776 +0000 UTC m=+4.111904448 container died 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-16T08:29:54.254 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 podman[100466]: 2024-03-16 08:29:54.052848328 +0000 UTC m=+4.456647999 container cleanup 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-16T08:29:54.254 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 bash[100466]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a 2024-03-16T08:29:54.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.a.service' 2024-03-16T08:29:54.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 podman[100480]: 2024-03-16 08:29:54.254016691 +0000 UTC m=+0.541702591 container remove 80728de252308569ca239f4289b47ed9544941dfe3deb3a9ebdc8eca756851f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:29:54.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.a.service: Deactivated successfully. 2024-03-16T08:29:54.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 systemd[1]: Stopped Ceph mds.a for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:29:54.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 16 08:29:54 smithi049 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.a.service: Consumed 1.108s CPU time. 2024-03-16T08:29:54.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:54.524 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-16T08:29:54.524 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-16T08:29:54.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.d 2024-03-16T08:29:54.787 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:54 smithi049 systemd[1]: Stopping Ceph mds.d for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:29:55.176 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d[72374]: 2024-03-16T08:29:54.839+0000 7ff549593700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:29:55.176 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:54 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d[72374]: 2024-03-16T08:29:54.839+0000 7ff549593700 -1 mds.d *** got signal Terminated *** 2024-03-16T08:29:58.783 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:58 smithi049 podman[100585]: 2024-03-16 08:29:58.521725161 +0000 UTC m=+3.773147755 container died 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:29:59.037 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:58 smithi049 podman[100585]: 2024-03-16 08:29:58.86534426 +0000 UTC m=+4.116766853 container cleanup 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T08:29:59.038 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:58 smithi049 bash[100585]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d 2024-03-16T08:29:59.038 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 16 08:29:59 smithi049 podman[100599]: 2024-03-16 08:29:59.024738232 +0000 UTC m=+0.502379208 container remove 76bfb26440d7c0205990aceb31b79cd04faddbb0c1eb0f69b6b24231378cbc9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T08:29:59.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.d.service' 2024-03-16T08:29:59.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:29:59.287 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-16T08:29:59.287 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-16T08:29:59.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.g 2024-03-16T08:29:59.537 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:29:59 smithi049 systemd[1]: Stopping Ceph mds.g for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:00.037 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:29:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g[72979]: 2024-03-16T08:29:59.635+0000 7fa1dc09c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:00.037 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:29:59 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g[72979]: 2024-03-16T08:29:59.635+0000 7fa1dc09c700 -1 mds.g *** got signal Terminated *** 2024-03-16T08:30:03.812 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:30:03 smithi049 podman[100707]: 2024-03-16 08:30:03.550077079 +0000 UTC m=+4.038912578 container died 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T08:30:04.136 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:30:03 smithi049 podman[100707]: 2024-03-16 08:30:03.910528594 +0000 UTC m=+4.399364093 container cleanup 053c28ba6dbacb97b4f483c47ad14297512cdc4736dc8ea4124b874fe9863b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T08:30:04.137 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 16 08:30:03 smithi049 bash[100707]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-g 2024-03-16T08:30:04.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.g.service' 2024-03-16T08:30:04.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:04.307 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-16T08:30:04.307 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-16T08:30:04.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.j 2024-03-16T08:30:04.788 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:04 smithi049 systemd[1]: Stopping Ceph mds.j for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:04.788 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j[72020]: 2024-03-16T08:30:04.622+0000 7f4fb1bac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:04.788 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:04 smithi049 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j[72020]: 2024-03-16T08:30:04.622+0000 7f4fb1bac700 -1 mds.j *** got signal Terminated *** 2024-03-16T08:30:07.948 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:07 smithi049 podman[100829]: 2024-03-16 08:30:07.67465251 +0000 UTC m=+3.149005719 container died a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:30:07.948 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:07 smithi049 podman[100829]: 2024-03-16 08:30:07.947887134 +0000 UTC m=+3.422240343 container cleanup a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T08:30:08.287 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:07 smithi049 bash[100829]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j 2024-03-16T08:30:08.287 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 16 08:30:08 smithi049 podman[100843]: 2024-03-16 08:30:08.190490284 +0000 UTC m=+0.512248792 container remove a7883aa1f873fc47e339071e441949221ac5bb9ba801ed6f109e268ee622dfd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-j, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:30:08.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.j.service' 2024-03-16T08:30:08.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:08.366 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-16T08:30:08.366 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-16T08:30:08.367 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.b 2024-03-16T08:30:08.678 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:08 smithi129 systemd[1]: Stopping Ceph mds.b for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:09.110 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: 2024-03-16T08:30:08.677+0000 7f4f842f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:09.111 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:08 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b[67038]: 2024-03-16T08:30:08.677+0000 7f4f842f0700 -1 mds.b *** got signal Terminated *** 2024-03-16T08:30:11.579 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:11 smithi129 podman[73479]: 2024-03-16 08:30:11.291704038 +0000 UTC m=+2.723583852 container died 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:30:11.862 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:11 smithi129 podman[73479]: 2024-03-16 08:30:11.794501908 +0000 UTC m=+3.226381711 container cleanup 253f5c42bc63d567fca04d9ff8e7aa55591a5ccddd12456e563de8c939f40769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:30:11.862 INFO:journalctl@ceph.mds.b.smithi129.stdout:Mar 16 08:30:11 smithi129 bash[73479]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-b 2024-03-16T08:30:12.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.b.service' 2024-03-16T08:30:12.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:12.173 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-16T08:30:12.173 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-16T08:30:12.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.e 2024-03-16T08:30:12.360 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:12 smithi129 systemd[1]: Stopping Ceph mds.e for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:12.860 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: 2024-03-16T08:30:12.481+0000 7f2672687700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:12.860 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:12 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e[66410]: 2024-03-16T08:30:12.481+0000 7f2672687700 -1 mds.e *** got signal Terminated *** 2024-03-16T08:30:15.565 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:15 smithi129 podman[73601]: 2024-03-16 08:30:15.297866178 +0000 UTC m=+2.913225995 container died de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-16T08:30:15.860 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:15 smithi129 podman[73601]: 2024-03-16 08:30:15.664544175 +0000 UTC m=+3.279903962 container cleanup de493ff268004cc99ffe825d96f0c90f8cb6168b279ce1dfa876c7bb9734c751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T08:30:15.861 INFO:journalctl@ceph.mds.e.smithi129.stdout:Mar 16 08:30:15 smithi129 bash[73601]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-e 2024-03-16T08:30:16.076 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.e.service' 2024-03-16T08:30:16.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:16.111 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-16T08:30:16.111 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-16T08:30:16.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.h 2024-03-16T08:30:16.610 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:16 smithi129 systemd[1]: Stopping Ceph mds.h for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:16.610 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: 2024-03-16T08:30:16.434+0000 7f0d00978700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:16.611 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:16 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h[66717]: 2024-03-16T08:30:16.434+0000 7f0d00978700 -1 mds.h *** got signal Terminated *** 2024-03-16T08:30:19.610 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:19 smithi129 podman[73721]: 2024-03-16 08:30:19.303545919 +0000 UTC m=+2.978650044 container died 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:30:19.860 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:19 smithi129 podman[73721]: 2024-03-16 08:30:19.667901197 +0000 UTC m=+3.343005312 container cleanup 51399ec5619a940b50c31da5a6bfbb49f8120c1e2d7f3b9ef5a73fef7b477ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T08:30:19.861 INFO:journalctl@ceph.mds.h.smithi129.stdout:Mar 16 08:30:19 smithi129 bash[73721]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-h 2024-03-16T08:30:20.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.h.service' 2024-03-16T08:30:20.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:20.124 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-16T08:30:20.124 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-16T08:30:20.124 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.k 2024-03-16T08:30:20.610 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:20 smithi129 systemd[1]: Stopping Ceph mds.k for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:20.610 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k[66089]: 2024-03-16T08:30:20.437+0000 7f3995cc6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:20.611 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:20 smithi129 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k[66089]: 2024-03-16T08:30:20.437+0000 7f3995cc6700 -1 mds.k *** got signal Terminated *** 2024-03-16T08:30:24.713 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:24 smithi129 podman[73842]: 2024-03-16 08:30:24.445354837 +0000 UTC m=+4.109125192 container died 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T08:30:25.064 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:24 smithi129 podman[73842]: 2024-03-16 08:30:24.804576385 +0000 UTC m=+4.468346737 container cleanup 880418fc63dbb5bcb57b0574d136621bfa10175e82b0e401e84af3341a27c619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T08:30:25.064 INFO:journalctl@ceph.mds.k.smithi129.stdout:Mar 16 08:30:24 smithi129 bash[73842]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-k 2024-03-16T08:30:25.225 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.k.service' 2024-03-16T08:30:25.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:25.259 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-16T08:30:25.259 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-16T08:30:25.259 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.c 2024-03-16T08:30:25.569 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:25 smithi154 systemd[1]: Stopping Ceph mds.c for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:25.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: 2024-03-16T08:30:25.567+0000 7fb9ffe5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:25.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:25 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c[67670]: 2024-03-16T08:30:25.567+0000 7fb9ffe5f700 -1 mds.c *** got signal Terminated *** 2024-03-16T08:30:27.646 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:27 smithi154 podman[74663]: 2024-03-16 08:30:27.301281258 +0000 UTC m=+1.843870900 container died 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-16T08:30:27.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:27 smithi154 podman[74663]: 2024-03-16 08:30:27.768177535 +0000 UTC m=+2.310767185 container cleanup 0c3ad7ae175ba04677aa71841d12a7018faca8d8509bea986f152f1bd9e72a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-16T08:30:27.975 INFO:journalctl@ceph.mds.c.smithi154.stdout:Mar 16 08:30:27 smithi154 bash[74663]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-c 2024-03-16T08:30:28.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.c.service' 2024-03-16T08:30:28.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:28.180 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-16T08:30:28.181 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-16T08:30:28.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.f 2024-03-16T08:30:28.726 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:28 smithi154 systemd[1]: Stopping Ceph mds.f for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:28.726 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: 2024-03-16T08:30:28.504+0000 7f496a2a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:28.726 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:28 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f[67366]: 2024-03-16T08:30:28.504+0000 7f496a2a0700 -1 mds.f *** got signal Terminated *** 2024-03-16T08:30:31.613 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:31 smithi154 podman[74795]: 2024-03-16 08:30:31.303177059 +0000 UTC m=+2.901376420 container died d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T08:30:31.613 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:31 smithi154 podman[74795]: 2024-03-16 08:30:31.613358431 +0000 UTC m=+3.211557793 container cleanup d354d091e09e7226aa9073ecd1183fadd3d2fc4ddb93e14a9577775d94ee1457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T08:30:31.864 INFO:journalctl@ceph.mds.f.smithi154.stdout:Mar 16 08:30:31 smithi154 bash[74795]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-f 2024-03-16T08:30:32.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.f.service' 2024-03-16T08:30:32.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:32.061 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-16T08:30:32.061 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-16T08:30:32.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.i 2024-03-16T08:30:32.475 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:32 smithi154 systemd[1]: Stopping Ceph mds.i for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:32.476 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: 2024-03-16T08:30:32.358+0000 7f44f6e8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:32.476 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:32 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i[67989]: 2024-03-16T08:30:32.358+0000 7f44f6e8e700 -1 mds.i *** got signal Terminated *** 2024-03-16T08:30:35.725 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:35 smithi154 podman[74915]: 2024-03-16 08:30:35.290933587 +0000 UTC m=+3.018043796 container died fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T08:30:36.101 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:35 smithi154 podman[74915]: 2024-03-16 08:30:35.917135212 +0000 UTC m=+3.644245420 container cleanup fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T08:30:36.101 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:35 smithi154 bash[74915]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i 2024-03-16T08:30:36.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.i.service' 2024-03-16T08:30:36.367 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:36 smithi154 podman[74929]: 2024-03-16 08:30:36.101488133 +0000 UTC m=+0.800081493 container remove fc38ddb5e94146fa380b99d80c62a16e4dbb4805619c4ad622825bb0ee9e90d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-i, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307) 2024-03-16T08:30:36.367 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:36 smithi154 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.i.service: Deactivated successfully. 2024-03-16T08:30:36.368 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:36 smithi154 systemd[1]: Stopped Ceph mds.i for 053f2308-e36c-11ee-95c9-87774f69a715. 2024-03-16T08:30:36.368 INFO:journalctl@ceph.mds.i.smithi154.stdout:Mar 16 08:30:36 smithi154 systemd[1]: ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.i.service: Consumed 3min 52.800s CPU time. 2024-03-16T08:30:36.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:36.379 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-16T08:30:36.380 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-16T08:30:36.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.l 2024-03-16T08:30:36.712 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:36 smithi154 systemd[1]: Stopping Ceph mds.l for 053f2308-e36c-11ee-95c9-87774f69a715... 2024-03-16T08:30:36.975 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l[67056]: 2024-03-16T08:30:36.711+0000 7fa2f54ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:30:36.975 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:36 smithi154 ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l[67056]: 2024-03-16T08:30:36.711+0000 7fa2f54ba700 -1 mds.l *** got signal Terminated *** 2024-03-16T08:30:38.062 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:37 smithi154 podman[75034]: 2024-03-16 08:30:37.767459872 +0000 UTC m=+1.169137802 container died 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T08:30:38.392 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:38 smithi154 podman[75034]: 2024-03-16 08:30:38.062573025 +0000 UTC m=+1.464250950 container cleanup 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T08:30:38.392 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:38 smithi154 bash[75034]: ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l 2024-03-16T08:30:38.393 INFO:journalctl@ceph.mds.l.smithi154.stdout:Mar 16 08:30:38 smithi154 podman[75048]: 2024-03-16 08:30:38.280313558 +0000 UTC m=+0.502106410 container remove 222958ac3969436cf95d6d4718d9be03ead3bd8b085c62ed4f82400269a048de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-053f2308-e36c-11ee-95c9-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-16T08:30:38.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-053f2308-e36c-11ee-95c9-87774f69a715@mds.l.service' 2024-03-16T08:30:38.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T08:30:38.445 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-16T08:30:38.445 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T08:30:38.649 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:41.205 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T08:30:41.395 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:43.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T08:30:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:46.478 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:30:46.517 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:30:46.553 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:30:46.581 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-16T08:30:46.584 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi049/crash 2024-03-16T08:30:46.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash -- . 2024-03-16T08:30:46.634 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T08:30:46.634 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-03-16T08:30:46.635 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi129/crash 2024-03-16T08:30:46.636 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash -- . 2024-03-16T08:30:46.734 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T08:30:46.734 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-03-16T08:30:46.736 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi154/crash 2024-03-16T08:30:46.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash -- . 2024-03-16T08:30:46.774 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/053f2308-e36c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T08:30:46.774 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-03-16T08:30:46.775 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-16T08:30:46.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-16T08:30:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:2024-03-16T08:09:02.237965+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T08:30:46.819 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-16T08:30:46.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[SEC\]' /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-16T08:30:46.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]' /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-16T08:30:46.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[WRN\]' /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-16T08:30:47.054 INFO:teuthology.orchestra.run.smithi049.stdout:2024-03-16T08:09:02.237965+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T08:30:47.054 INFO:tasks.cephadm:Compressing logs... 2024-03-16T08:30:47.054 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T08:30:47.096 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T08:30:47.099 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T08:30:47.124 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T08:30:47.125 INFO:teuthology.orchestra.run.smithi049.stderr:: No such file or directory 2024-03-16T08:30:47.125 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-16T08:30:47.125 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.6.log: /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-16T08:30:47.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-16T08:30:47.139 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-16T08:30:47.139 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T08:30:47.139 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-16T08:30:47.139 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T08:30:47.139 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.7.log: 90.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.e.log: 94.7% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-16T08:30:47.140 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T08:30:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T08:30:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T08:30:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-16T08:30:47.141 INFO:teuthology.orchestra.run.smithi049.stderr: 94.7% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-16T08:30:47.141 INFO:teuthology.orchestra.run.smithi049.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:: No such file or directory 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.i.log: /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-16T08:30:47.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr: 94.7%/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log: -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.2.log: 87.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-16T08:30:47.145 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-16T08:30:47.148 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.y.log: 91.8% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-16T08:30:47.148 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log 2024-03-16T08:30:47.148 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T08:30:47.148 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T08:30:47.150 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log 2024-03-16T08:30:47.150 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T08:30:47.150 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log: 84.2% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T08:30:47.150 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-16T08:30:47.150 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-16T08:30:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.9.log: 87.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T08:30:47.151 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log: 87.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T08:30:47.151 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-16T08:30:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T08:30:47.164 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-16T08:30:47.164 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T08:30:47.165 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T08:30:47.168 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-16T08:30:47.169 INFO:teuthology.orchestra.run.smithi049.stderr: 94.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-16T08:30:47.197 INFO:teuthology.orchestra.run.smithi129.stderr: 94.5% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-16T08:30:47.198 INFO:teuthology.orchestra.run.smithi154.stderr: 94.7% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-16T08:30:47.198 INFO:teuthology.orchestra.run.smithi154.stderr: 92.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-16T08:30:47.200 INFO:teuthology.orchestra.run.smithi129.stderr: 92.7% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-16T08:30:47.221 INFO:teuthology.orchestra.run.smithi154.stderr: 94.0% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-16T08:30:47.459 INFO:teuthology.orchestra.run.smithi049.stderr: 89.8% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-16T08:30:47.476 INFO:teuthology.orchestra.run.smithi129.stderr: 92.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-16T08:30:47.515 INFO:teuthology.orchestra.run.smithi154.stderr: 91.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-16T08:30:47.619 INFO:teuthology.orchestra.run.smithi129.stderr: 91.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-16T08:30:48.376 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-16T08:30:48.650 INFO:teuthology.orchestra.run.smithi049.stderr: 92.0% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-16T08:30:48.832 INFO:teuthology.orchestra.run.smithi049.stderr: 91.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-16T08:30:49.203 INFO:teuthology.orchestra.run.smithi154.stderr: 92.8% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-16T08:30:49.243 INFO:teuthology.orchestra.run.smithi129.stderr: 91.9% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-16T08:30:49.702 INFO:teuthology.orchestra.run.smithi129.stderr: 92.1% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-16T08:30:49.752 INFO:teuthology.orchestra.run.smithi154.stderr: 92.1% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-16T08:30:49.990 INFO:teuthology.orchestra.run.smithi129.stderr: 92.7% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-16T08:30:50.119 INFO:teuthology.orchestra.run.smithi049.stderr: 92.1% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-16T08:30:50.166 INFO:teuthology.orchestra.run.smithi154.stderr: 92.3% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-16T08:30:50.187 INFO:teuthology.orchestra.run.smithi129.stderr: 92.4% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-16T08:30:50.189 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-03-16T08:30:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m3.076s 2024-03-16T08:30:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m11.201s 2024-03-16T08:30:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.390s 2024-03-16T08:30:50.375 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-16T08:30:50.377 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-03-16T08:30:50.377 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m3.264s 2024-03-16T08:30:50.377 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m11.229s 2024-03-16T08:30:50.377 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.388s 2024-03-16T08:30:50.624 INFO:teuthology.orchestra.run.smithi049.stderr: 91.6% -- replaced with /var/log/ceph/053f2308-e36c-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-16T08:30:50.625 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-03-16T08:30:50.626 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m3.517s 2024-03-16T08:30:50.626 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m11.023s 2024-03-16T08:30:50.626 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.324s 2024-03-16T08:30:50.626 INFO:tasks.cephadm:Archiving logs... 2024-03-16T08:30:50.626 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi049/log 2024-03-16T08:30:50.627 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T08:30:52.396 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi129/log 2024-03-16T08:30:52.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T08:30:54.192 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi154/log 2024-03-16T08:30:54.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T08:30:56.008 INFO:tasks.cephadm:Removing cluster... 2024-03-16T08:30:56.008 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force 2024-03-16T08:30:56.213 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:56.484 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force 2024-03-16T08:30:56.675 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:56.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 053f2308-e36c-11ee-95c9-87774f69a715 --force 2024-03-16T08:30:57.178 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 053f2308-e36c-11ee-95c9-87774f69a715 2024-03-16T08:30:57.491 INFO:tasks.cephadm:Removing cephadm ... 2024-03-16T08:30:57.491 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T08:30:57.511 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T08:30:57.527 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T08:30:57.542 INFO:tasks.cephadm:Teardown complete 2024-03-16T08:30:57.543 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T08:30:57.555 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T08:30:57.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T08:30:57.558 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T08:30:57.570 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T08:30:57.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T08:30:57.697 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T08:30:57.697 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:30:57.700 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:30:57.703 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:30:57.714 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:30:57.715 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:30:57.716 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 +4995us[+4995us] +/- 110ms 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -2105us[-2112us] +/- 57ms 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -317us[ -317us] +/- 66ms 2024-03-16T08:30:57.719 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +4984us[+4987us] +/- 110ms 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv02.front.sepia.ceph.com 3 6 377 0 -2115us[-2111us] +/- 57ms 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -341us[ -338us] +/- 66ms 2024-03-16T08:30:57.721 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:30:57.722 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:30:57.722 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-16T08:30:57.722 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 0 +4973us[+4973us] +/- 110ms 2024-03-16T08:30:57.722 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -2126us[-2081us] +/- 57ms 2024-03-16T08:30:57.722 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -338us[ -293us] +/- 66ms 2024-03-16T08:30:57.723 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:30:57.723 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T08:30:57.773 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T08:30:57.773 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T08:30:57.793 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T08:30:57.890 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T08:30:57.964 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T08:30:58.036 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:30:58.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:58.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:30:58.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:58.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:30:58.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:58.135 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T08:30:58.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T08:30:58.163 INFO:teuthology.task.internal:Duration was 1935.903114 seconds 2024-03-16T08:30:58.164 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T08:30:58.204 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T08:30:58.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T08:30:58.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T08:30:58.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T08:30:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:30:58.251 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:30:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:30:58.757 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T08:30:58.757 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-03-16T08:30:58.757 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T08:30:58.780 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-03-16T08:30:58.780 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T08:30:58.804 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-03-16T08:30:58.804 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T08:30:58.831 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T08:30:58.831 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T08:30:58.834 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T08:30:58.847 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T08:30:58.900 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T08:30:58.900 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T08:30:58.909 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T08:30:58.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T08:30:59.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T08:30:59.314 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T08:30:59.314 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T08:30:59.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T08:30:59.371 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T08:30:59.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T08:30:59.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.439 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-03-16T08:30:59.441 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-03-16T08:30:59.464 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-03-16T08:30:59.493 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:59.515 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:59.529 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T08:30:59.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:30:59.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T08:30:59.559 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T08:30:59.560 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi049 2024-03-16T08:30:59.560 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T08:30:59.608 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi129 2024-03-16T08:30:59.608 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T08:30:59.646 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274/remote/smithi154 2024-03-16T08:30:59.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T08:30:59.690 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T08:30:59.690 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T08:30:59.693 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T08:30:59.695 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T08:30:59.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T08:30:59.751 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T08:30:59.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T08:30:59.774 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T08:30:59.775 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T08:30:59.777 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T08:30:59.779 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T08:30:59.789 INFO:teuthology.orchestra.run.smithi049.stdout: 265300 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 08:30 /home/ubuntu/cephtest 2024-03-16T08:30:59.791 INFO:teuthology.orchestra.run.smithi129.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 08:30 /home/ubuntu/cephtest 2024-03-16T08:30:59.793 INFO:teuthology.orchestra.run.smithi154.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 08:30 /home/ubuntu/cephtest 2024-03-16T08:30:59.794 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T08:30:59.815 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T08:30:59.841 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T08:30:59.867 DEBUG:teuthology.nuke:shortname: smithi049 2024-03-16T08:30:59.867 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T08:30:59.889 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.351409', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnormpttfCKGWCKUOjckB27FFkBp2gjD+7haduh3HIhRdkrSE1tbj2Ur2LS2lD3F3ZYUDgAtPUf8+Z+v8+0RRY='} 2024-03-16T08:30:59.894 DEBUG:teuthology.nuke:shortname: smithi129 2024-03-16T08:30:59.894 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T08:30:59.919 DEBUG:teuthology.nuke:shortname: smithi154 2024-03-16T08:30:59.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T08:30:59.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.352525', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGub6YpicLRQ9cvdCDyu3L7E6X5nFxm4jOB7H8/H+kGiOMncWimwLT0XUrHYjFP0z6PiWpuaeuSWDoDR1tlZu00='} 2024-03-16T08:30:59.956 INFO:teuthology.orchestra.console.smithi049:Power off 2024-03-16T08:30:59.957 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T08:30:59.968 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602274', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 07:48:03.350200', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAZwaH2Q3LVkDBssky8FHxrHr/lqrLej7urkItChXsSmRfNhRY1nF/YDmTkXs+VAISpeWJg0ngFWVebfTTKx7uo='} 2024-03-16T08:30:59.980 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-03-16T08:30:59.980 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:00.002 INFO:teuthology.orchestra.console.smithi129:Power off 2024-03-16T08:31:00.002 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T08:31:00.017 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-16T08:31:00.025 DEBUG:teuthology.orchestra.console.smithi129:power off output: Chassis Power Control: Down/Off 2024-03-16T08:31:00.026 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:00.044 INFO:teuthology.orchestra.console.smithi154:Power off 2024-03-16T08:31:00.045 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T08:31:00.054 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-03-16T08:31:00.066 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-03-16T08:31:00.066 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:00.090 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-16T08:31:04.019 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:04.055 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:04.091 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:04.130 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-16T08:31:04.166 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-03-16T08:31:04.201 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-16T08:31:08.134 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:08.167 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:08.202 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:08.244 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-03-16T08:31:08.277 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2024-03-16T08:31:08.312 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-16T08:31:08.346 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-03-16T08:31:08.378 INFO:teuthology.orchestra.console.smithi129:Power off completed 2024-03-16T08:31:12.314 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T08:31:12.424 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-03-16T08:31:12.525 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-03-16T08:31:12.630 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} duration: 1935.9031138420105 failure_reason: '"2024-03-16T08:09:02.237965+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-16T08:31:12.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:31:12.720 INFO:teuthology.run:FAIL